Machine learning to analyze dietary intake

A team of Tufts researchers is developing an artificial intelligence incorporated with computer vision techniques to report nutrient intake.
Headshots of the five members of the Picture This! team
Clockwise from top left: Ph.D. candidate Shreyas Kamath, Graduate Education Dean and Professor Karen Panetta, postdoctoral scholar Eleanor Skonkoff, Research Assistant Professor Erin Hennessy, and Professor Christina Economos.

Dietary intake is key to human health and disease risk across the globe, but accurate and reliable assessment of dietary intake has remained elusive in the nutrition science field.

An interdisciplinary team from Tufts School of Engineering and the Friedman School of Nutrition and Policy has now taken aim at developing a method to provide an accurate account of an individual's food and nutrient intake. Based on digital imaging of food, artificial intelligence techniques, and computer vision, the team's methodology was designed to increase accuracy, minimize cost, and enhance usability. The prototype algorithm uses digital images to automatically identify food and provide an estimate of calorie consumption, and machine learning to classify foods.

The Picture This! team includes electrical engineering Ph.D. candidate Shreyas Kamath and Dean of Graduate Education for the School of Engineering and Professor Karen Panetta, working alongside their Friedman School colleagues, Professor Christina Economos, Research Assistant Professor Erin Hennessy, and postdoctoral scholar Eleanor Skonkoff.

The team placed second in a recent Elevator Pitch Contest organized by Sight and Life Foundation. Along with six fellow finalists, the team won a trip to the American Society for Nutrition's annual meeting to pitch their proposal to experts and potential investors. The contest sought "disruptive ideas to stimulate networks and dialogue, especially among innovators, to improve existing approaches to measure nutrition," and judges were particularly looking to identify and support low-cost and precise analytical tools that could be implemented and produce a meaningful impact relatively quickly.

Picture This! was also a finalist in the 2018 Tufts Food and Nutrition Entrepreneurship Competition.