Retell Lecture

Computer scientist Shwetak Patel and his team are developing new sensing systems. The initial focus was really around energy and water monitoring. They built a new generation of smart sensors that monitor electronic interference on a home’s power line or water pressure changes in the plumbing. Most of this technology has already found industrial applications. And Patel and his team turned their attention to adapting the technology for personal health monitoring. “So how do we take this noise and make it into a signal of interest? That was a kind of the core of what we did for many years. And we’re taking that work and applying it to other domains.” They’re looking to take advantage of all the functionality built into our smart phones. With the users’ permission, this App can use the microphone, built into most smart phones, to listen to background noises, such as coughing, searching for patterns that suggest a trip to the doctor might be an order. “We construct these models that try and understand how sound works, how it what its patterns are? And we give it a whole bunch of examples of different kinds of audio, things like people talking, things like people laughing, sneezing and of course coughing.” This App uses a phone’s camera to check haemoglobin levels in blood by analysing the colour of capillary fluids through the skin.“Generally what happens is if you’re anemic your bloods are going to be a little less red, and we take advantage of that by putting your finger over a camera of a phone, the camera of the phone can actually see the coloration of the blood.”And this test uses the camera tell parents worried about jaundice in new-born infants. “Now jaundice is something that doctors who have seen tons of babies, it just can figure out on a very basic level of; is this baby, do they need get treatment or are they in a good condition? Whereas a first-time parent has no idea necessarily what jaundice might look like.” The researchers say the built-in sensors found in smart phones are already commonplace but their applications and their implications for our health and well-being maybe more far-reaching than we ever imagined.