Nutrify AI Grant Application
Take a photo of food and learn about it.
- Demo Nutrify web app (primitive)
- iOS app video demo (1 min, cool)
- Founder intro video (1 min)
- Slide deck
Hello, Nat, Daniel, Evan, Alex and anyone else reading this,
We're Daniel and Josh Bourke, brothers from Australia and the founders of Nutrify.
This is our AI Grant application.
What is Nutrify?
Nutrify is a (very) early stage food education application.
Short version
Take a photo of food and learn about it.
Longer version
Nutrify is a nutritionist/chef/dietician in your pocket.
It allows you to take a photo of food and learn about it.
Food being a food label or food item.
For example, take a photo of a banana and get its nutritional breakdown and more.
Or take a photo of a food label and find out what food additive "222" is (sodium hydrogen sulphite, a preservative).
Got a couple of spare ingredients in the fridge?
Take a photo of them and let Nutrify prepare recipe ideas for you.
As an analogy, you can think of Nutrify as a search engine designed specifically for food.
Or our favourite: a Pokédex for food.
Note: The longer version describes a near future version of Nutrify (something the AI grant will help to fund).
Why Nutrify?
Food labels are confusing.
Obesity is increasing around the world.
Consumers are up against multi-million dollar food marketing campaigns.
Of course not all of these are bad.
But nutrition education, even in medical schools (some medical students report receiving less than 2-hours nutrition education per year) is lacking.
Nutrify aims to provide a way for people to change this.
Starting with: take a photo of food and learn about it.
Why the camera?
Simplicity is paramount.
And nutrition is complicated.
So anymore than one step to learn more and it's over.
And taking a photo of food and learning things about it is fun.
Learning about food doesn't have to stop at knowing that 100g of banana has 20g of carbohydrates - that's only one dimension.
Imagine taking a photo of an apricot and learning that the word "apricot" comes from the Latin "precocious" because the fruit ripens early. And the letter "a" appears to be added by a mistake in translation.
Bertrand Russell said this kind of "useless" knowledge only makes the fruit sweeter.
And we agree.
Now what other secrets are out there?
Nutrify plans to find them and help others find them too.
Who are we/why us?
- Daniel Bourke, machine learning engineer/instructor/nutritionist/food scientist/farmer
I (Daniel) graduated with a food science and nutrition degree (mostly for my own interest) but then started learning machine learning (and teaching it at the same time online), before getting hired as a machine learning engineer.
I now teach machine learning to 100,000+ students in 185+ different countries and run a YouTube channel with 125k+ subscribers dedicated to machine learning, health and education.
But it's time to combine my loves for food and AI via Nutrify.
- Joshua Bourke, iOS developer/lifeguard
I (Joshua) started teaching myself to build iOS apps a couple of years ago.
And after building and publishing a couple of games on the App Store, I started building an app to track my Brazilian Jiu Jitsu results called MyBJJ. Many people at the gym I train at are now using it and I plan to launch it to a larger audience soon.
As for Nutrify, I like to make apps and I like food.
Competition
Visual and text search engines already exist through Google, Bing, Pinterest.
However, these services offer width (search everything) rather than depth (search one topic and make it fun).
Nutrify will differentiate itself with depth rather than width.
Focusing on food and food only.
There are also existing food education and fitness apps already such as MyFitnessPal, though these applications focus specifically on tracking rather than education.
Nutrify will differentiate itself from existing static tracking apps by being an interactive education app.
Roadblocks
Food has an extremely long tail.
And while Nutrify will likely work very well for common, whole foods (e.g. a whole apple versus an apple pie), dishes and other niche food items will be a challenge.
Though we believe we'll be able to leverage the power of recent innovations in vision-language models as well as generative models like Stable Diffusion to handle and generate training data for rare occurrences.
On a longer term horizon, cooking is arguably a harder problem to solve than self-driving because it uses five human senses rather than two.
Roadmap
- Months 1-6 - Build out functionality, cover 500-1000+ whole foods and 99.99% of ingredients on food labels.
- Month 7 - Supermarket test: walk into a random supermarket and Nutrify should be able to identify and offer insights to all food items in the fresh produce section as well as any food label.
- Months 7-12 - Productize and monetise Nutirfy, the goal will be to go to market as soon as possible (to gather data from real-world use), however, once the foundational functionality is in place, the next thing will be making sure it's worth paying for (a prototype is nice but a reliable system is better).
To begin, Nutrify will start as a standalone app on web/iOS (this is where our specialities are).
Later on it may progress into something like an API that can be integrated into apps such as Snapchat for access to a larger audience.
What we'll do with the grant funds if we win
We already have plans to work on Nutrify for the next year at least (self-funding) whenever we have time but the AI grant will speedup this journey significantly.
- $250k cash - With this we'll both be able to work on Nutrify full-time for at least a year, so progress will be quite fast, if we pay ourselves ~$75k/year each, we'll have plenty to spare for unseen circumstances.
- $250k Azure credits - Most of this will likely go towards GPU compute (8x A100 GPUs for a year on Azure is ~$110k). Though I'd like to see how far we can go with just fine-tuning pretrained models. Hosting will be almost free due to most of compute happening on device (network latency doesn't make for a fun time).
- $10,000 in human-labelling credits from Scale.ai - I (Daniel) have a feeling that most of the labelling will be done by large vision-language models (or self-supervised) but we'll use these credits for creating gold-standard ground truths to compare auto-labels to.
Thank you
Thank you for reading our application and feel free to contact us if you have any questions.