AI Talk: All about pictures

July 24th, 2020 / By V. “Juggy” Jagannathan, PhD

I was not looking to create a themed blog this week, but it turns out I found three different articles all related to pictures!

Generate pictures

GPT-2 is a powerful language model created by OpenAI for ingesting millions of documents. If you feed it some text, the model generates a coherent completion of the seed text. This model was created in February 2019. Now, you get to witness the new incarnation of the same model. GPT-2 was trained on text: this new model is trained on images from the ImageNet dataset of 14 million images. More correctly, this new model is trained on pixels! A two-dimensional image is converted to a single dimension of pixels, similar to a stream of characters/words. The new model is called iGPT and if you feed it part of the image it completes the image. Pretty impressive generation capabilities. Wonder what it would do if only the top half of a face covered with a mask is given: Will it correctly identify a person? Given the COVID-19 situation, it is rather irritating to note that facial recognition does not work for logging into your phone. For the technically inclined, you can read this paper on the subject that won an honorable mention for best paper in the last week’s International Conference on Machine Learning.

Sun pictures

On February 9, 2020, the Solar Orbiter was launched. This was a joint collaborative mission between the European Space Agency (ESA) and NASA with the explicit goal of studying the dun. The spacecraft is equipped with a range of instruments specifically designed to photograph the sun. One such instrument is an Extreme Ultraviolet Imager (EUI) and it has been sending in stunning images of the sun from roughly halfway between Earth and the sun. This MIT Technology Review article provides a good summary of the images that we now get and their import. The scientists are super excited about receiving the images and are hoping to solve the mystery of why the atmosphere of the sun is so much hotter than its core. I have no clue about the science here, but the images are pretty cool!

Hazy pictures

I keep getting news about by alma mater—the Indian Institute of Technology, Madras—though it has been more than four decades since I graduated from there. One of these news items caught my attention last week. Professor Rajagopolan and his research team there have devised an image post-processing algorithm to remove haze from pictures. So, if there is a haze in a nighttime picture, the algorithm makes the image clear. Similarly, the algorithm can be applied to underwater images. This got me wondering: Why hasn’t this been done before? If you can do this with images, why not do it with video? And, if you can do it with video, imagine driving down a fogged-up road, and your navigation display provides a crystal-clear view of what is in front of you. Surely, this must be in our future. Tesla, Ford, Chrysler: Do you have this in your future product suite? Sort of similar to noise cancellation in headsets, this is fog cancellation in video streams!

I am always looking for feedback and if you would like me to cover a story, please let me know. “See something, say something!” Leave me a comment below or ask a question on my blogger profile page.

V. “Juggy” Jagannathan, PhD, is Director of Research for 3M M*Modal and is an AI Evangelist with four decades of experience in AI and Computer Science research.