Have you ever wanted to know how to prepare a dish just using photos you’ve seen on Instagram or Pinterest? Have you just seen some delicious-looking chocolate chip cookies on the Internet and you now want to know the ingredients you’ll need and the main steps to make them? This is now possible thanks to an experimental artificial intelligence project called Pic2Recipe.

Pic2Recipe is an application that analyses the ingredients and the steps to cook a dish by just using a photo. It is being developed by engineering students from the Massachusetts Institute of Technology (MIT), in collaboration with the Technical University of Catalonia (UPC). For the moment, the program is in its very first stages of development, but thanks to its artificial intelligence system, it can already predict ingredients and recipes with up to 80% accuracy.

The key to Pic2Recipe’s success is its huge database, which is essential for it to work correctly. This database has been obtained by combining the Food-101 Data Set, a food identifier algorithm developed by Swiss researchers in 2014, with a million-recipe database called Recipe1M, designed by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Recipes have also been added from two key recipe websites in the United States, All Recipes and Food.com. In this way, Pic2Recipe manages a greater degree of precision in linking the images to be deciphered and their ingredients and the recipes used.

It is certainly a brilliant idea, although like all computer programs, it has several glitches. For example, research has concluded that it is more effective for baked goods, such as muffins and cookies, than for dishes such as shakes and sushi, which pose far more problems for the identification software. Pic2Recipe also has problems coming up with recipe results for foods that are cooked in several different ways in the database.

Another similar project is one being carried out by the New York Institute of Electrical and Electronics Engineers (IEEE), which analyses a photo to monitor the amount of carbohydrates in a dish, in order to help people suffering from diabetes to check how much carbs they are consuming. In the long run, IEEE researchers will be working to analyse a photograph to show the calories, fats, proteins and micronutrients contained in the recipe photographed.

You can read the original piece on Techcrunch.