An Augmented Reality Microscope for Cancer Detection

An Augmented Reality Microscope will help in detecting cancer cells in upcoming days

With each passing day virtual reality, augmented reality and machine learning are opening new pathways in varied medical disciplines. It has been found that the application of the machine learning in varied medical departments specifically the radiology, ophthalmology, dermatology and pathology can help in deriving accurate and precise reports of the patients on global scale which can in turn boost the overall health sector on global level.

Google research team has come with an innovative and promising Augmented Reality Microscope, which has started showing remarkable results in detecting the breast cancer symptoms with highest rate of accuracy found only in very few trained pathologists.

This augmented reality microscope can perform a wide range of task like offering visual feedback, in form of the heat maps, contours, arrows, animation as well as some texts. As it is being powered by the machine learning it has got the ability to perform some problem solving as well as like going through smart object detection, classification as well as the quantification.

Google researchers have developed the augmented reality microscope based on the modern computation models along with the deep learning models using TensorFlow. The use of the machine learning algorithm helps in getting the output right on the screen in real time.

Detecting breast cancer based using new algorithms

Google researchers has trained this augmented reality microscope to run two very different cancer detection algorithms in order to detect cancer with highest rate of accuracy.One algorithm allows the augmented reality microscope to detect the breast cancer metastases present in the lymph node while the other algorithm trains it to detect the prostate cancer present in the prostatectomy specimens.

As it is able to take snaps in higher magnification of 4-40x, it allows the physician or researchers to have a look at the growing tumour regions with the green contour right on their screen. Furthermore these detailed imaging of the cells also allows the doctors or pathologist to carefully study the area under the influence of tumours cells without any issues.

Future application of the augmented reality microscope

Google is hopeful that augmented reality microscope will find its due place in the global health sector market along with making a huge impact in the same. It will be a great asset for the developing countries where it becomes extremely difficulty to detect the common infectious disease like tuberculosis and malaria in the early stages.

Apart from being robust in nature it is also quite light weight which will help it in finding it easily adoptable in wide varieties of industries globally apart from the pathology.

Secondly augmented reality microscope will also be finding a huge success in the digital industry here it can be used to create digital workflow prints which aren’t’ possible to be done by the modern scanners.

Healthcare will certainly be the prime focus of the Google in establishing, introducing and marketing the augmented reality microscope in the upcoming days but tech giant is also looking forward to bring this revolutionary device to the life research and material science disciples as well.

Article: Augmented Reality on the Web, for Everyone

Google wants to bring augmented reality to browsers and web pages

Google seeks to bring augmented reality technology (AR) to a web platform in the name called 'Article'. On Monday, the company published advances in the performance of a 3D prototype in both mobile device browsers and desktop computers. The objective of this series of tests is to expand access to this technology and advance the development of browsers that in the future are enabled to support AR.

According to a publication on the official blog, millions of Android and iOS devices will be able to provide AR experiences from their browsers in the coming months. This means that to experience this technology a user would not need a specific application but with the browser open you can visualize digital objects pointing to real places through your cell phone. Through Google's 'Article', the user could access a page with AR contents and view them from their desktop computer, in the office or at home.

Until now to use a game or watch a content in augmented reality had to pass two assumptions: that we had an app pre-installed on our mobile or download a specific way to see it. We also have the disadvantage that not all AR experiences work the same, so we can easily install a few of them.

Google wants to break that barrier with a very good idea: to introduce this function in a web browser. In this way, all we need is to have a Chrome app, open a web page, click on a button and have the camera open so we can see the object in question. At the moment it is a project in development but as a proposal to make this technology more accessible it can be very successful.

Article, as Google has called it, is a 3D object viewer that will be integrated into the browser. When we see 'Article' we can interact with it and spin it, for example, to see how 'Article' looks on each side. This is not all, we will have a dedicated button in 'Article' to take us to the camera and we can place the model in question in front of our eyes.

So explain the Mountain View when we activate the option of augmented reality in 'Article' will take us to the camera and we will see a white ring where a few seconds later the object in question will appear. In the demo of 'Article' that they have shown in the blog it is seen that this function is very intuitive and easy to use.

For augmented reality it will be an important step because of the agility it will bring to use it without any intermediary in the form of additional application. In addition, with the quality they have today any Android phone is enough to offer a basic but satisfactory experience.

At the moment, the 'Article' is only a prototype and it seems that we will soon see it integrated into the browser that we use daily to visit web pages. What they have facilitated are the tools to encourage content developers to create their own objects and test Article's capabilities.

Google ARCore Brings Augmented Reality to Android

Google’s Innovative Software Development Kit

Google has released an innovative software development kit – SDK known as ARCore in order to bring about an experience of augmented reality – AR nearer to Android users. The ARCore system being a software development kit will enable anyone to bring the AR features to the prevailing Android phones with any hardware updates. Google had mentioned in a statement recently that the platform has now been made available for the developers for the purpose of experimenting which would reduce the competences of augmented reality to the prevailing and the forth-coming Android phones.

 The company constructed on important technologies utilised in Tango, another AR platform by Google, though ARCore is said to be scalable all over the ecosystem of Android since it does not need any extra hardware. Google has mentioned that ARCore branches from its work on Tango. When it had been introduced in 2015, Tango enabled users with compatible devices to create detailed 3D maps of the world by directing a camera towards an object. The Lenovo PHAB2 Pro had been the first Tango phone which had been released.
 

Work with Java/Open GL….

 
The innovative AR platform is said to be rolled out to Google Pixel together with Samsung S8 that tends to run on Android 7.0 Nougat and more. Google has been functioning with manufacturers like ASUS, Huawei, LG, and Samsung as well as with the other main smartphone makers for scrutiny on quality and performance. ARCore that tend to work with Java/OpenGL, Unity and Unreal technologies is said to focus on three features namely motion tracking, environmental, understanding and light estimation.

These would permit the camera of a phone to identify feature points in a room, detect horizontal surfaces where AR object could be located and interpret lighting conditions in order that virtual objects could be lit properly. The company has also manufactured `Blocks and Tilt Brush application to create 3D content to be utilised in AR apps. Apple had make public its ARKit platform directed on augmented reality applications during its annual Worldwide Developers Conference enabling developers to design apps which would take virtual content at the top of the scenes of real-life.
 

New Visual Position Service

 
Google had mentioned in a blog post that they have been targeting 100 million devices towards the end of the preview. Moreover the company has also added experiments with AR page portraying what the ARCore technology can perform. This move of Google into augmented reality is not surprising wherein the firm had mentioned at its developer’s conference this year that it had been working on a new visual position service which is said to support phone in understanding where people are indoors.

 And this can be done be being proficient in detecting what objects are place before the camera of a device. The iOS 11 of Apple that would be made available to the consumer later in the year will comprise of the ARKit which, as per the announcement at the conference would enable iPhone and iPad developers to experience the combination of digital overlays on the real world.

Microsoft Patent Reveals That The Company is Working on Augmented Reality Wand

Microsoft’s Patent – Wand-Like Controller

From Microsoft’s patent filing, it has now come to light that the company has been functioning on a wand-like controller for augmented reality (AR) and `HoloLens’. Microsoft had made an announcement of a new patent on August 22 of the design for an augmented reality input device wherein the wand-like controller appears like a cross between a motion controller and a pen, according to the report on The Verge.

The software giant company had originally filed the patent in June 2016 on the same day it had published its Windows Mixed Reality vision video. In the video one would see what seems to be a wand-like device in action being utilised to articulate hand tracking with a HTC Vive VR headset.

 There is a possibility that this tends to be simply a reference device which had been testing with partners by Microsoft though it could also hind that the company intends accessories which would operate across several VR headsets.

The software major had intended to introduce a range of Mixed Reality headsets from partners like Asus, Acer, Dell, HP and Lenovo, later in the year. The headsets would be available at a price below the prevailing VR devices, with the unit of Acer beginning at $299.

Aiming - Six Degrees of Freedom

Microsoft has been aiming on six degrees of freedom tracking with the need of traditional external sensors positioned all through a room.

 The lowest specs for Mixed Reality platform of Microsoft also specify most PCs would have the potential of running the most basic apps together with games. Augmented reality has been gaining power quickly in the US, powered by Snapchat Lenses and Facebook Stories.

 Facebook has also been operating on emerging augmented reality glasses which could balance digital object with that of the physical world. Mark Zuckerberg, the CEO of the company had earlier shown photos of comparable glasses stating that they would be the future of augmented reality though had not disclosed that it would be developing this kind of product.

CEO of Facebook is of the view that augmented reality would be the subsequent big user tech platform after the smartphones. This information had been reported after the announcement of Microsoft on its Build developer conference in May that the company plans to utilise AI for the purpose of safety in workplaces.

Purpose – Match with Promotional Video

It had also showcased how AI can make for example a construction site to be safer and also efficient. Spotted by WalkingCat, the patent revealed an accessory which seemed to appear like want, having buttons, a trigger and a finger guard.

It is unknown at the moment as to what the accessory would be capable of doing though it seems like it could be utilised as a pointing device for the purpose of presentations, a prop for augmented reality games or could be also a painting tool.

The purpose of the device tends to match with promotional video wherein Microsoft portrays too small want controllers which tend to interact with an augmented reality setting. It appears similar to Google’s Draw accessory for Google Glass and increases on the controls that are available with the HoloLens clicker.

Researchers Use Augmented Reality to Turn Coloring Books into 3D Experience

Colouring App Developed – Disney Research

A colouring app developed by Disney Research could cause characters to spring from the page in a 3D wonderthrough augmented reality. A child tends to colour a character on the book page normally while a smartphone or tablet running the app tends to monitor the drawing.

Based on the colouring of the child, the app fills the colours in real time on a 3D animated version of the character which is visible on the screen of the device and integrates it into a video. The core focus on the traditional activity of the colouring is maintained by the app while providing the magical digital overlay which tends to improves the engagement.

User testing performed by adults rather than the children in the early study, researchers observed that most of the users informed that the app increased their enthusiasm to draw in colouring books while 80% stated that the app increased their feeling of linking with the character.

Disney researchers, ETH Zurich as well as the Swiss university EPFL presented the augmented reality app at the ISMAR 2015 – IEEE International Symposium on Mixed and Augmented Reality in Fukuoka, Japan.

Disney Color & Play

Though the work of the research was presented to scientific audience, it had gone through the tech transfer process already, motivating the commercial product known as `Disney Color and Play’ which was launched earlier in the year by Disney Publishing Worldwide and Bendon.

The work tends to fit in a huge initiative known as Augmented Creativity at Disney Research that aims on utilising augmented reality in improving creative play. Robert W, Sumner, principal research scientist leading the group on animation and interactive graphics at Disney Research stated that `augmented reality holds unique and promising capabilities in bridging between real world activities and digital experiences enabling user in engaging their imagination and boost their creativity.

They are thrilled to have the opportunity in presenting the scientific advances behind this technology and are particularly happy that it is available to consumers, thanks to the cooperation with Disney Publishing’.

App Functions on Device With Camera

The researchers, inorder to create a new experience, at first created animated 3D virtual characters and then utilised custom software in generating 2D line art representations of the characters for colouring book.

 The app functioning on a device with a camera that viewed the user and the colouring book automatically detects the character the user tends to colour, displaying the 3D version. As the child progresses with the colour to the 2D drawing, the app tends to apply the same colour to the 3D characters in the areas visible in the 2D drawing as well as to the remainder of the 3D form which is not visible in the book.

 Owing to the colouring occurring in real time, the illusion is developed that the user is also colouring the blocked areas with the same texturing of the colour. Defining how to apply colour to the blocked areas seems one of the most difficult issue, according to Sumner.

By mirroring the user’s strokes on colours does not seem to work since the patter of colours used for a character’s face would not be the same for the back of the character’s head. The colour also needs to be continuous in order that no seams are seen between the visible areas and the blocked areas or where dissimilar portions of the textures tend to meet.