Artificial Intelligence offers many opportunities for Middle School and High School students to create socially impactful projects. In this blog we answer common questions about how to use AI for social good, and also share a talk by Dr Manjari Jain - an accomplished biologist - who talks to kids about how AI and Machine Learning can be used to study nature.
What is AI for Social Good?
AI for social good is a term given to projects that use different AI techniques to solve environmental and societal problems and assist humans in various ways. Examples are AIs that can detect wildfires or predict the direction of wildfires, AIs that can identify plastic in the ocean and protect marine life, AIs that can help rural communities diagnose diseases or find clean water, etc.
How can AI be used for social good?
AI algorithms are very good a detecting patterns, whether they be patterns in pictures, in text, or any other kind of data. This ability can be harnessed by a very large number of social good projects. In the example above - the AI that detects wildfires can be trained by satellite or drone images to tell the difference between an area with a wildfire and an area without. Once trained, this AI can be used to automatically detect wildfires and wildfire spread.
What Social Good AI projects can Middle School students do?
Middle School students studying with AIClub have done the following social good projects that are great examples for other kids. Mihit built TomatoGenius - an AI that can detect the health of tomato plants and help farmers diagnose diseases in their crops. Anika built an AI to detect Diatbetic Retinopathy from eye scans, and Shriya built an AI to help the deaf communicate using ASL. Middle school students can do many projects to help society and the environment by using Deep Neural Networks and Image Classification.
What Social Good AI projects can High School students do?
AIClub high school students have also done many socially impactful projects. Check out Abhinav’s project that uses advanced AI - Transformers and BERT - to detect whether tweets contain evidence of disasters. Anay built an AI to recommend nutritious food to users and help them eat healthier. High School students can do advanced projects using powerful software like PyTorch and TensorFlow, and build AIs to classify images, process text and natural language, and more.
Dr Manjari Jain’s talk
In the video below, Dr Manjari Jain talks about how she uses AI and Machine Learning to detect bird calls and classify different types of birds from the sounds that they make. This type of AI can be used for everything from nature education to tracking migratory bird patterns to studying climate change from changes in bird behavior.
Transcript below:
Now welcome Dr. Manjari Jain. She is a very, very good professor. And she studies how birds and insects communicate. And you know what she is doing? She is using her knowledge of how birds with the songs and the sounds communicate. And she is using AI to figure out what they are saying. Isn't that cool? Would you like to hear about her? Yeah.
Alright. So great. I'm just looking at the kids just to assess the age class. I just want to say hello to every one of you. So can you just briefly switch on your video? Okay, hello, everyone. I am Manjai and like Supriya said I work on birds. And I'm really looking forward to this.
Okay, so I just want to share the joy of my work with all your kids in the hope that some of you will then decide to work on something like this, when you grow up, or even now, because you can do it. I mean, you can start working on things that I talked about today, right now, you can start from tomorrow, okay, so it's not that difficult. And that's why I hope that you will enjoy the talk and ask me lots of questions, you can send me the emails, or you can drop me a note right now and ask me your questions.
So I want to tell you how we can study nature, using machine learning. Okay, this whole, your entire program is on artificial intelligence. And I will explain to you how it can help us to study nature, okay. And that's what I do. I study nature. And I collaborate with engineers, and the people who use artificial intelligence for various other reasons, but I collaborate with them so that we can together study nature using machine learning. So I'll just explain to you how we can listen to the world around us and use that to understand the world around us and with the help of machine learning.
So this is just to show you my lab. These are the students in my lab and this is my daughter, and she also works with us, she's five years old. So that should tell you that each one of you can also please visit us in our lab at ICER. Mohali, and you can work with us, intern with us for a few months if you want if your parents allow you.
Okay, so just very briefly, what is artificial intelligence, basically artificial intelligence, you can think of it as a bigger circle, okay, in which there are many different kinds of things people are trying to do, but the larger aim is to create what we call as intelligent machines, right? And why are these machines called intelligent machines? Because they are being made to be like humans, okay. So we know that humans have a lot of special features such as thinking, you know, the ability to think to sense that many other animals also have this. But we are trying to simulate the machines to be like humans who can think who can sense who can adapt.
Machine learning is only a subset of artificial intelligence, okay, so it's like an application of AI if you like, okay and what does one do with machine learning basically it is learning using allowing the machine to learn to help us solve some problems to put it very simply, basically, you give it a lot of data or things observations that you have made, you put all of that data into the machine, okay. And then without actually programming the machine to learn machine learning the concept has been designed in such a way the whole idea is such that just given the patterns in the data, the machine will pick up these patterns without you having to tell it Okay, it will pick it up and then it will be able to do a lot of problem-solving for you.
Okay, so this is by and large, the broad framework of machine learning so it is simply humans programming machines to function like humans but more efficient.
I want you all to see what birds I don't know how many of you know any birds? Do any of you know any birds? You can unmute, I suppose, and just say if you know any birds, not necessarily in this panel. Does anyone know any birds? Okay, yes. Yes, yes. sparrows. Wow, who's that? Amazing. So that's great. You already know so many so now when you go out, right? You have when you look at the birds you have to figure out okay, which bird is this? Okay, for instance, this one right and you can see my mouse pointer right here the arrow so through you know that you mark this out and look at it. Yes, absolutely. Can you mark this one out for me the yellow one here right.
I don't that is an Oriole but this is a parrot now if you look at the colors, they're somewhat similar like yellow, but the moment you see this you are green. So you know that these are different. Like you said you looked at the shade of the color. So now if there is a lot of sunlight, the green may actually that's why I put this picture in particular, it might actually start to look like you know, but you can make the difference. The subtle differences you are being able to pick up right you know, this is green and not yellow, right? Yes, and it has a red because what else? Let's think of this bird. Hmm, the one down below. This is a sunbird. It's a purple sunbird. Okay. And I want you to tell me if I want you to explain to me what the bird looks like. How would you explain to me because you have seen the bird and you are telling me Hey, I saw this bird. You know it. seen this bird? I'm not actually seeing this. So the distinctive features you see your brain is already doing it for you. Your brain is already telling you which is the most important feature Okay, here. You did not say that it has black eyes. Please note it has black eyes, right. But we did not. Your brain did not say that it has black eyes because that is not really to your brain. It is not such an important character to begin with. Right? Now this bird. If someone had to someone else can I ask someone else to try? What are the most special thing about this bird? describe this bird to me.
Now you all have to remember all of this that you just said. Okay, and then we will see what machines can do. Okay. So now when we are identifying these birds, to begin with, it might seem to you actually it is more of a problem for adults. Children are very smart. You know, it's only the big old people who somewhat you know, struggle a lot to figure this out. But you know, if you ask children because you are so uninhibited, you don't have any barriers in your mind. Right? You immediately think about it in a very natural way. That is exactly what we want. Okay, so how do we identify the birds? This is what we do, we first go out, okay, we go out in the wild and we start The birds Okay, here, there's a bird up in the tree. Okay. And then we start looking at various important features of the bird, let's say, size. Now the pictures that I showed showed to you, I'm not showing you the actual size difference between the birds. But when you actually go out in the wild, you can see the trees this big, and as compared to the tree, the bird was the small Okay, so you know the relative size and you can also compare the size of unknown bird to some known birds, for example, this could be the size of a sparrow, you all know the minor, the brown color bird who makes a lot of chattering sound outside the dark brown birds with orange eyes. Yeah, everybody knows crows, for sure. Okay, so you know that these birds are of different sizes. So you can use that as a reference to say, the bird was smaller than the Crow, but bigger than a minor. Or you could say for this one, it was bigger than the Crow but smaller than an eagle. An eagle is a really big bird. Okay, so that could be the feature that you're looking at with respect to size. What else? Just like you all had said? How else do we notice about birds we examine other features. For example, the color of the body someone said red cheeks, right? So or you can see overall brown in color. This bird is overall Brown, is it black, green, white? No, it's overall brown and color. The shape of the beak Okay, so is it long income like the purpose and But no, it is short, but it is curved? You can see that this is curved right? It's not a blunt beak. Right? Was it found near water? Was it found on a tree was it found on the ground this is an important feature to note because this can help you understand which bird It is okay, so if you don't know the bird if you note these things, this can help us tell you which bird it is. Some birds will never be found on the ground. Some birds will never be found on the trees. There are some birds who are usually found near water Okay, so these things can help you figure out which bird it is now I'm pleased remember I'm telling you how humans do this exercise okay.
We are trying to figure out how us humans you and me when we go out to do some birding How will we identify a bird? Okay, so this is how we do our identification when we see a bird.
But you all know there is something special about birds that birds are really vocal they make a lot of sound you know, you know that many birds often you will wake up with the chattering of birds and it can be annoying sometimes. We know that birds make sound okay, they are vocal and that's the other way that we can find birds. We can find birds by their sound okay. So what we can do these guys are simply recording the calls of the birds Okay. And then they will look at the calls of the birds are also listening to note that they have got this headphones on right. So they are listening to what bird is singing and they are recording it so that they can look at it more carefully later. So even if you have not seen a bird you may have heard it and that can help you identify and I'll give you some examples here. Okay, don't worry about these patterns, these patterns just if imagine that the picture of the board was not there. If we had only heard the bird then when record them these patterns are like the identity of the bird. By these patterns you can tell by listening to it and looking at the patterns you can tell which bird it was.
And that is again our brain telling us based on some patterns of the sound that that is a Crow. Okay, now this is again what this graph is don't worry about that all I want you to see is that you know these patterns of the sound look very different this is what the sound of a Crow will look like if you see it recorded and see it in the software. So when we record the sounds okay, and then we plot it in a certain software we can see these patterns so now you can see the sound remember the Batman movie here where the Batman is basically using ultrasound to look at the map of the house right something like that can be done for the sound of the birds okay. So, you can look at the sound of the birds and you can figure out okay, well which bird is this okay.
So, what can machines do for us? So, something as simple as that you can think of and that is what the machine learning algorithms are trying to extract out of the data you feed but how does it know? So, to begin with, just like how we begin you have to have a learning phase okay or a training phase? You have to go out regularly with your birding friends or you can come out with me and we can go out birding and you will tray we will train you. So, what happens in the learning phase and the same thing we do for machines because remember we are trying to simulate human behavior, right? So we teach the machine that hey, this set of data is a crow and that is not a crow. Something as simple as that. And then you can have something more fancy. This is a crow and this is a crow of this variety, okay? So something more specialized you can do okay.
So first you train the algorithm or the machine with the data that you tell what it is okay, so now the machine knows that okay, all of these say for instance, pictures of crows. Then we have to tell which features are important just like you said, a pointy head, the shape of the beak, the color of the body. These are some important features for birds, but when you're doing This kind of exercise for say for instance identifying mountains, there is no shape of the beak there might be some other feature right. So, you have to tell the machine what are the important features and it uses all of that okay and then basically converts this data that okay the she said that the this these pictures are of cruise these pictures are not of clothes and these features are important. So, let me let me study what are the features of this set of pictures Okay. And then it will see well these are the features of the set of pictures and these features are lacking in these set of pictures okay. So, now it has made a function or that equation okay. Using which now, when you throw data at it, when you do this for a very long time with a very large data set it has trained that means learning is complete.
And that is how you can use machine learning for various different kinds of problem-solving. Okay, so now let going back to the bird example. If you look at various features, size was one feature color of the body is a feature number of tail bands is a feature habit is a feature. And the data that we are pumping in is okay, well, the color of the body was brown. Okay, so we gave it brown, right? shortened code on the tree. So for every feature, we gave it an answer, okay. And these kinds of features are called categorical features don't have to worry about it. Because they can belong two answers can belong to distinct categories, it is either red, or it is blue, it cannot be somewhere in between, right? So these are categorical variable, they could be numerical variables, where the actual number length of the beat can be in some centimeters. Okay, so the I'm just telling you what kinds of features there can be there could be yes or no answers it was it found on the ground? Yes. found on the ground? No. Okay. So that would be a binary feature. So various kinds of features then is used. And based on that the prediction emerges.
Now what kind of questions machine learning can handle? Don't get worried about this, that this sounds too complicated. I will tell you what, exactly classification. So I have given a lot of cards, okay, of pictures. And I take pictures of various animals. And I tell you, well, which are the birds, which are the frogs, which are the fishes put them separately? Can you do that? We can do that, of course. And that is the kind of things machines can do.
That was lovely. Thank you. Dr. Jain. That was amazing. I also want to quickly introduce you to Dr. Nisha, she's the my partner who's organizing it from the USA. Hello,
thank you so much for your presentation. I really appreciate it.
Two questions, you know, are there also data sets available for sound in case somebody wants to make a sound identification project number one and number two If the children are to actually go out and collect sound, Bird sounds, how much data would they need on one of these child-friendly platforms to be able to train it? Realistically?
Yeah. So while there are several databases for bird calls specially because just birds are so gorgeous and fascinating. So many people all around the world are working on birds and they have uploaded the data of bird sounds on various libraries. For example, in Cornell mecool, a library of Cornell University is there you can look it up. And we can download sounds from there, okay, there's tears, birds, birds sound from all over the world. Then there is something called xeno Canto, and I can send those links to you Supriya later with in an email or something. So is there no campus there, which also has recordings from all over the world and it is recorded, not just by scientists, these are all recordings done by citizens. In fact, most of the these recordings come from citizens. Okay, so these we call them citizen scientists. And again, age no bar for this, like even younger kids, if they can, and what do we need to record sound of birds? One, of course, all our devices, the mobile phones that we have, they are optimized for human sound. And birds have a large part of their frequency range actually overlaps with our own hearing, right. That's why we can hear them, unlike bats, right. So if you want to record birds, the cheapest thing is borrow the phone of your parent and just go out and record there are various apps. I mean, I am sure the kids know more than me. Now various apps that you can download to record birds on your mobile phone. And you can upload it to these libraries, okay, so you can be a citizen scientist. Also, India also has various sound libraries. So there's I, there is India biodiversity portal in which you can upload pictures, you can upload sound. For insects, there is orthoptera species. I mean, there's so many there are so many. So it's just like when you go to the world of sound, it is a completely very large world out there for you to explore. And all you need is at this stage, just a mobile phone in front of my phone is very expensive. You can just buy these dictaphones they are small recorders which journalists carry no tiny recorders, which will cost you hardly a couple of 1000s maybe 2000 3000 rupees, okay, less expensive than an iPhone that belongs to your father or mother. So you can take those dictaphone and record his birds, the only thing is that you must be quiet when you're recording the birds. Because the birds are really soft. They're not harsh, like us. Right? So that is one and how much data do we need. So it depends upon what kind of classification you want to do if we want to just learn how to so what I showed today was on sound how we are classifying sound. And this sound basically is converted into images, which are those plots that I showed you, the spectrograms. And so in that sense, it becomes a little more like a few extra steps are added. Right? So you can just download all of these sounds from these libraries. And you can start some very basic classifications like is this a bird sound or not? Or is this sound of a wind of wind? Or sound of rain? Or at least Can you tell what is not a bird sound? Something like that something at the very, you know, basic level zero, we can start with and then see how where we go. Does that help you? Thank
you so much. Actually, if you don't mind I have a really short follow up question. So yes, Nisha, do you you know feed kind of sound into a sound specific
AI or do you did you say you convert sound to sort of images? Or do you convert sound to like for example more of its frequency, you know, spec values? Yeah,
you do all three? Yes. So I'll tell you so basically, to view the sound to see the sound you need certain kinds of software and these are freely available software one of the most common software again developed by Cornell Lab is Raven, our AV n everyone can download it for free and you can you know just record a sound and upload it you know, view it open the file and view it in Raven it'll help you see the sound and that is what I was showing you all of those were even a plots okay. And you can see the sound you can also hear it back right. So that is the first step the algorithms will be you know, programs that you will write to extract features from those sound okay, which is what in some sense Raven is doing anyway, it is telling you, what is the frequency or the pitch of the sound and how long that sound is for example. So it lasts for quite One time or that is a shorter sound, right? So the duration of the sound these features can be extracted and you can do it through Raven itself. Or if you want to go a little higher up, then you write your own code to make that plot
Wonderful, thank you so much. Really, really appreciate it and thank you for organizing this.