Humansare blessed with an extra sense of perception. A faculty of subtle awarenessnot explicable which involves perception of constantly changing environment butalso its relation to ourselves.
It involves the ideaof receiving information without using any of the other five senses; nor is ittaken from any experiences from the past or known. Sixth Sense freesinformation from its confines by seamlessly integrating it with reality, andthus making the entire world your computer. 1.1 Sixth Sense in theearly times: Mae’sMIT group of a few graduate students came up with an idea to integrate thedigital world into the human world and get easy access to information withouthaving to put in too much effort like taking out the phone and searching theinternet – they made a wristband that would read the Radio FrequencyIdentification tag to find out some information. For example, this wristbandcould read the RFID tag of a book that the user is holding in his hand.They also had a ring that used infrared to communicatewith the smart shelves in supermarkets that would give information aboutproducts. If suppose we pick some item from the market, the ring would glow redor green indication some information about the product depending on whatevercriteria we program into the system.
The motive was to make information moreuseful to people with minimal effort without any behaviour changes. Thewristband was a good idea, but it was not completely satisfying the main motive.Afterthis, a larger projector was mounted on a helmet for displaying information,but this became problematic since if the user would turn to talk to a friend,the data would project on their face. Figure 2 1.2 Sixth sense in therecent times: Thesixth sense device is switched to a neck piece which is smaller and moreefficient. It consists of a pocket projector, a mirror and a camera. The technology is based onrecognition of hand gestures, capture-process and manipulation of images.
The user wears a neck piece which has aprojector and camera linked to a mobile computer like a phone or such device inthe users’ pocket. Basically, the projector projects on any surface, it carriesthe information stored in it and it also collects information from the web.In the end, the user behaves like a computer and their hands and fingers arelike keyboard and mouse.Theprototype was built from an ordinary webcam and a battery-powered 3M projector,with an attached mirror — all connected to an internet-enabled mobile phone.The projector helped display all information on any surface. Few colouredmarker caps (red, blue, green, yellow) were worn in 4 different fingers thathelped in recognizing hand gestures.2 WORKING OF SIXTH SENSE TECHNOLOGY 2.1 COMPONENTS:The hardware componentsthat make up the sixth sense device are:- Camera- Projector- Colour Markers- Mobile device2.
1.1 Camera: The camera is used tocapture and recognize objects in view and track the user’s hand gestures usingcomputer-vision based techniques.Theimage/gesture captured is sent to the mobile device.
The camera recognizes everythingaround the user and with the help of the micro projector, it can display theinformation on any surface, even on the user’s hand. 2.1.2 Projector: Therole of the projector is that of interaction and sharing. The projectorcontains a battery inside, with 3 hours of battery life. The projector projectsinformation by using different surfaces as interfaces. The action of projector is to make it looklike the user is projecting information by touching something. Atiny LED projector is used to display data sent from the smart phone.
2.1.3 Colour Markers: These are placed at the tip of the user’s fingers.Marking the user’s fingers with red, yellow, green, and blue tape helps thecamera recognize gestures. The movements of these coloured markers areinterpreted into gestures that act as instructions for the projectedapplication interfaces. 2.1.
4 Mobile Device: Amobile device is used to transmit and receive data anywhere and to anyone viathe mobile internet. The mobile in the sixth sense device helps connect to theinternet and access information from it. It processes the image sent to it andprojects the necessary information. 2.2 HOWDOES IT WORK? · Sixth Sense device is a pendant likestructure which is like a mobile wearable interface.· It has a camera, a mirror and a projectorand is connected wirelessly to a Bluetooth or Wi-fi network on a mobile device.· The camera captures images and gestures.
· Information from the camera is sent to the mobiledevice for processing.· The projector projects the output image onto any desired surface.· Thus, digital information is freed fromits confines and placed in the physical world All the components are joined together to form a neckpiece.
The camera is used to recognize individuals, images and gestures that ismade, and the projector’s role is to assist in projecting the information onany surface. Coloured caps are used on the user’s fingers to identify thegestures and to make it easier for the software to differentiate fingers. Thesoftware analyses the data caught by the camera and tracks the locations ofcoloured markers by using computer vision techniques.
Any number of handgestures can be used provided they are all reasonably identified anddifferentiated for the system to understand and interpret it. This is possibleonly because the Sixth Sense device supports multi-touch and multi-userinteraction.Sixthsense technology gives the idea of not only what someone is interacting withbut also how they are interacting with it. The role of the software is tosearch the internet for information related of some object that is relevant tothe situation and then project it with the help of the projector.Thereare 3 kinds of gestures that this software recognizes: ü Multitouchgestures – it is where the user touches the screen and makes a certainapplication move by pinching or dragging. For example, in Map application.ü Freehand gestures – usual gestures used, likethe one used to take a picture.
ü Iconicgestures – drawing an icon in the air. The technology is mainly based on hand gesturerecognition, image capturing, processing, and manipulation. The user cannavigate a map displayed on any surface using hand gestures by a mapapplication. Users can zoom in or zoom out of an image using hand movements.Drawing application lets users draw on any surface by tracking the fingertipmovements. 3 TECHNOLOGIESUSING SIXTH SENSE AS A PLATFORM Thevarious technologies which has been using sixth sense technology as itsplatform are:- Radio Frequency Identification (RFID)- Whirlpool Washing Machines 3.1 RFID Sixth sense gives a platform for RFID based intelligencethat combines RFID events with information from other systems and sensors tomake inferences about people, objects, workspaces and their interactionsautomatically.Radio Frequency Identification is an electronictagging technology that allows detection and tracking of tags and thus theobjects that they are affixed to.
The sixth sense gadget can select a producteither by image recognition or radio frequency identification (RFID) tags andproject information. Theidea of Sixth Sense is to use Radio Frequency Identification technology alongwith a bunch of other enterprise systems that tracks user activity. Sixth senseincorporates algorithms that gives a range of undifferentiated tags andautomatically infers a range of information based on the observations. Thetechnology can automatically differentiate between people tags and object tags,learn the identities of people, infer the ownership of objects by people, learnthe nature of different zones in a workspace (e.g.
, private office versus conferenceroom), and perform other such inferences.By combining information from these diverse sources,Sixth Sense records all tag-level events in a raw database. The inferencealgorithms consume these raw events to infer events at the level of people, objects,and workspace zones, which are then recorded in a separate processed database.These databases can be polled together by running SQL queries. 3.2 WASHINGMACHINES Whirlpoolwashing machines is a remarkable application that incorporates the use of sixthsense technology. Whirlpool’s 2009 range of washing machines have thistechnology integrated in them.
Sixth sense in washing machines provide moreoptimisation of resources and is also efficient in terms of energy, time andwater. It can detect stubborn stains and adjusts wash impact. The sixth sense washing machines helps to protectthe environment and reduces electricity bills. They are energy efficient andare capable to adapt their performance to better suit the users’ needs.
AllWhirlpool washing machines using sixth sense technology works on threeprinciples: Sense, control and adaption, to ensure optimal performance. 4 APPLICATIONS Thesixth sense device has a variety of applications. The following are a few ofthose applications:· Making a call· Map application· Checking time· Creating multimedia reading experiences· Drawing application· Zooming features· Getting product information· Getting book information· Getting flight updates· Feed information about people· Taking pictures· Checking email 5 SIXTH SENSE TECHNOLOGY IN RETAILING Thereis always something that triggers our senses to connect to something particularin a store.
It may be a flash of colour, a soothing sound or a powerful scent. Retailersare working hard to make the customers happy and satisfied with what they’ve gotin store.Mobileapplications and digitization are reshaping the way retailers engage withcustomers and how customers prefer shopping. IBM Watson has been working withmajor retailers such as North Face and Macy’s to empower their shoppingexperiences with visual recognition and sixth sense to enable more personalizedservice. They use in-store cameras and sixth sense to monitor the movement ofproducts and then use the information to optimize promotions.
‘Affectiva’, anMIT backed company uses facial recognition technology to analyse human facesand can sense up to 7 human emotions., including anger, contempt, sadness, joy,surprise, fear and disgust and 20 facial expressions, age, ethnicity andgender. This database provides valuable insights to retailers about customerpreferences. Thecamera in sixth sense device hasthe ability to simultaneously scan and analyse hundreds of points of a person’sface. It can also decipher between human objects and non-human. Via aninstantaneous recognition of facial detection, emotion detection, demographicand feature detection, computer vision is layering in context to who isinteraction by adding in the how and the why.
5.1 IMPLEMENTATIONS This can be implemented in everyarea of retailing:• Theuser can pick up a product in shop (e.g. dishwasher liquid), and the systemcould display related information back on the product itself. • Thesystem can recognize any book picked up by the user and display its rating,about the author, synopsis on the book cover. It can also display additionalinformation such as reader’s commentsand also if any of their friends have read it.• Thesystem is able to recognize people by their appearances and project a wordcloud of related information retrieved from the internet on the person’s body.
• Integratedcredit card processing after sale.