On this episode of How We Made That App, embark on a captivating journey into STEM education with host Madhukar Kumar and the Co-Founder of the app Numerade Alex Lee! Alex gives an in-depth and fascinating fusion of philosophy and technology that propels Numerade's innovative learning platform. Alex unveils the intricate layers of AI and machine learning models that power their educational ecosystem
On this episode of How We Made That App, embark on a captivating journey into STEM education with host Madhukar Kumar and the Co-Founder of the brilliant app Numerade Alex Lee!
Alex gives an in-depth and fascinating fusion of philosophy and technology that propels Numerade's innovative learning platform. Alex unveils the intricate layers of AI and machine learning models that power their educational ecosystem. Beyond the present, he explores the promising future integration of Large Language Models (LLMs), offering a glimpse into the next frontier of education.
Numerade is more than just AI and LLM enhancements, Alex emphasizes the human touch woven into Numerade's approach. Discover the impact of meaningful interactions on the learning experience and the deliberate efforts to maintain a personal connection in the digital realm. Alex envisions growth by seamlessly aligning Numerade's services with the dynamic advancements in AI, creating a bridge between cutting-edge technology and genuine human engagement.
Tune in as this episode unravels the philosophy, technology, and human-centric approach that define Numerade's quest to revolutionize STEM education.
Key Quotes:
Timestamps
Links
Madhukar: [00:00:00] Hello, and welcome to How We Made That App. I'm your host, Madhukar Kumar. After starting my career as a developer and then as a product manager, I somehow ended up becoming the Chief Marketing Officer at SingleStore. And today, we have with us Alex Lee, co founder of Numerade, a virtual learning platform that adds more meaning back to STEM education.
This is straight off of your website. Alex, so welcome, Alex.
Alex: Thank you for having me, Madhukar.
Madhukar: Alex, you know, one of the questions that I get asked almost every day and most often is, how do you pronounce your name? So I'm going to ask that question to you. How do you pronounce your name, Alex?
Alex: Exactly that. 100 percent spot on. And it's probably like the easiest two syllables of a name you can [00:01:00] pronounce. And my last name is even easier, Lee. One syllable, three letters.
Madhukar: I'll put you in touch with my parents. Can you share the story of how you started Numerade? Actually, even before you do that, tell our audience what is Numerade, who uses it, and Who are your main users and customers?
Alex: So at Numerade, we exist to help students who are struggling with math and science. And what we do is we partner with our educator network of college professors, high school teachers, talented TAs to help us create these step by step video solutions for problems in math and science, problems that a lot of students generally have a lot of difficulty across the board.
Most of our students are high school students and college students, so they're kind of in that range of doing AP Calc. Maybe this is their first time taking an Intro to Calculus course, and therefore, Not having a lot [00:02:00] of experience with those topics before, they do have a lot of that's what we found, a lot of struggles and a lot of difficulties really understanding the content deeply.
And that's where Numerade comes in, is we provide these bite sized, really short form videos for students to watch and learn how to tackle certain parts, certain sets of skills when it comes to problem solving, when you're, for example, doing a Physics question on momentum, we'll have an expert educator be able to guide you through the steps needed to solve that problem successfully.
And our thesis has always been that when it comes to learning this complex material, it's so much more effective to be able to get sight and sound. So to be able to sit down, have in front of you an expert educator, again, who's walking you through the video, speaking to you. The high level concepts guiding you through the various skills that are required to be able to tackle that problem.
That's what we found really, really constructive to [00:03:00] the learning cycle for our students.
Madhukar: Wow, this is I wish this was around when I was in school because I struggled with Textbooks, personally, and especially around math as well as some of science. So, you mentioned bite sized videos. Was that inspired by my attention span?
Or are you a big fan of snack sized candy? Exactly.
Alex: And, you know, this is really where I think we've been able to differentiate from a lot of the other education platforms out there. Unfortunately, the state of education and the state of how education products are built these days is more for administrators and for purchasing and procurement officers for large districts.
And so the products that students end up using aren't really custom tailored for them. For us, our philosophy was always trying to build the best product, the best learning experience for students. Numerade [00:04:00] was being developed, That's when TikTok was really cementing its preferred consumption habit and building out this really unique concept around short form video and providing that kind of bite sized pieces of content for students to consume.
And seeing that, we realized All of these education platforms, not to knock on Khan Academy or other platforms that have really excellent material. The fact is, though, that sometimes when it comes to, if there's a problem that you have and you just want to learn it, sometimes 10 minutes or longer, that's not what you're looking for.
And for us, really focusing on delivering. Experience to the student that helped them really tackle the exact question and the exact problems that they were having really helped. And so our philosophy has always been, let's build for the, build for the user first. Let's identify where their preferences are, and then build for that.
Madhukar: Got it. So, so it's, it's almost like TikTok for education, so to [00:05:00] speak. And does this go along with the textbook or these are more around specific topics and questions?
Alex: For both. So, students have the ability to open up their textbook and follow along on Numerade. They also do have the ability to go on to Numerade and to ask a question.
It could be really a wide range of topics. For nowadays, we cover pretty much the whole gamut of STEM, so anything from Again, calculus to organic chemistry to even topics like nursing which really do feature a lot of concepts in biology and chemistry. Students are able to ask those types of questions and get responses from our educators.
Madhukar: And typically how long are these bite sized
Alex: videos? The average length of a video is about two minutes. When we talk about the, yeah, the TikTok generation, the TikTok platform, it's very much mirrored after those consumer preferences. So I could
Madhukar: watch a two minute video at twice the speed and learn all about how the universe [00:06:00] was started?
Exactly. On a much more serious note, like how do you take like really in depth topics that require a lot of material to explain and bring that down to a concise two minutes? Is that what we would say is one of your key USPs as a company and as a platform? Yeah,
Alex: absolutely. And I think one thing that we do also in the background, and this is where AI and machine learning comes in, is being able to create holistic end to end experiences that Really stitch together a lot of these different videos to provide something that takes the student through the whole journey of learning something first at a conceptual level.
So really building that knowledge foundation. So, for example, understanding momentum even is. And then gradually as they're building that knowledge framework, we're Giving them more discrete items and problems for them to solve. In that way, we're doing more of [00:07:00] that skill building aspect. So, really honing in on how do you solve momentum questions and equations and whatnot.
What gave you
Madhukar: that idea of building something like this? And how, do you have students directly signing up? For Numerade or is it through the schools and the textbook companies?
Alex: We're completely direct to students, so if you're a student and you're struggling with a calculus problem tonight, you can go on to Numerade and subscribe as a user and get access to all of our content.
The inspiration for that was really actually from our previous experience of Running a different startup in the education space as well. And in a nutshell, there we were able to get a first hand look into some of how the best tutors in the world are tutoring their students. And we borrowed a lot from the frameworks that they were using.
And to answer your question about, you know, [00:08:00] how do we come up with this idea of really building in this? End to end learning journey for our students. It was really inspired by a lot of the great tutors that we were able to work with, you know, in a previous life.
Madhukar: And how do you, how do you look for very engaging educators who can boil down concepts in two minutes?
Do you look for both a deep educational background as well as Viral dance moves that they can combine, or do you go
Alex: directly into Absolutely. Every educator has to, as part of the interview process, give us a TikTok dance. But, you know, actually that's, that's a really important point where the most important piece actually comes down to the educators.
They are the ones who are creating the content, and they are the ones who are creating this engaging experience for our students. The way that we were able to, you know, Back in the early days, Recruit, our first cohort of educators, was actually [00:09:00] going to universities and identifying who are the really great TAs that are teaching calculus.
And there's plenty of resources online where you can go and say, oh yeah, I love this TA or I love this professor. And what we would do is we would look at those reviews and we'd go out and reach out to them say, hey, We have this new platform that's coming up and we'd love for you to participate in it.
And from there, we've been able to really scale that out and build a platform where a lot of these really naturally engaging educators are able to be a part of Numerade and contribute to that learning experience.
Madhukar: Moving over to the technical side of things and the architecture and so of how you built that app.
When did you start this? Was this during the pandemic days? Was it? Pre pandemic days? It
Alex: was pre pandemic days, yeah. So we were, we started building the platform around 2018 and officially launched around 2019 or so. Got
Madhukar: it. And are you able to talk about the growth and if that growth [00:10:00] Either flattened or increased during the pandemic and post pandemic, have you seen any changes?
Alex: Yeah, absolutely. During the pandemic, so much has been written about it. Absolutely, we saw tons of growth with students and with I think the need for services like ours. I think Even nowadays with the pandemic behind us, the need for continued support and, you know, when you look at the learning loss that exists today, you know, we're seeing so many reports around students being one, two, three grade levels behind where they used to be.
It makes our platform all the more important to be able to step in and provide some of those resources to address that. Those learning gaps.
Madhukar: How did you go about building it? Did it come out like, let's go build it out in React or Java or build it as a small app on mobile device and then build on top of that?
Or was it, no, let's sit down, let's think about the architecture [00:11:00] and then go build it out like a proper production ready system? Like, which direction did you go? Small and iterate? Or did you think through the architecture and then figure out what to do? Yeah,
Alex: I would say we, more on the small and iterate side.
Again, previously we did have a different startup in the tutoring space that was leveraging a lot of, really, at that time, newer technologies, particularly around WebRTC and being able to, off of that platform, these real time communication protocols. And that was one of the ways that we were able to really scale up the educator side and provide a platform for our educators to create content.
Very, very seamlessly. That was one of the bottlenecks that we initially ran into where educators would have trouble creating the content and uploading content, and that was when we had decided, let's actually use a lot of the tech that we have existing and port that over so that educators [00:12:00] can create all of these step by step, short form videos for us at scale.
So, really, it was partially bringing over a lot of the existing tech that we had from a different startup, and then massaging it to fit the use cases for these new kinds of users that we're bringing on. And you mentioned
Madhukar: WebRTC, so I'm assuming that obviously has improved so much since 2019, and that enabled you to have educators build videos using just a browser, which could be over a computer or even using their mobile devices.
Is that correct?
Alex: Exactly, exactly. That's an amazing explanation for a CMO.
Madhukar: That is just a fake title. In terms of the building of the application itself, do you face any data challenges? Like, how do you think about data? Is this in terms of how people consume your unstructured data? Or do you also use [00:13:00] that data with some structured data and then come up with analytics?
Use that to improve your overall engagement or even content. Yeah,
Alex: absolutely. And a lot of that data, too, feeds into AI and ML models that we've built to really streamline a lot of the operational aspect. But nowadays, with generative AI and with LLMs, it does feed into more student facing products as well.
In the early days, we did have quite a few challenges really building out our data infrastructure. And that's actually when we had our first first exploration of, at that time, single source memsql, and we're looking at how we could really build out and leverage a lot of the key, unique features around the memsql platform, particularly, you know, with it being able to do essentially a jack of all trades, you know, both analytical operations and also transactional.
Unfortunately, at that time, we weren't able to move forward with memsql. [00:14:00] We were kind of Piggybacking off the AWS offerings, and there wasn't really an easy plug and play solution there. But what we realized was we did need to have a more robust way of really orchestrating all of the behind the scenes data.
technologies, whether it was like our data warehouse, we're building up our data lake, and being able to stitch everything together to one, drive a lot of the machine learning and AI models that we were building at the time to support both, again, operational aspects of the company, but then also the student facing ones.
But then also for our For our product teams to really gain deeper insights into how our users were interacting and behaving with the platform. And to do that at scale, it required a rethinking of how we approach data in the first place.
Madhukar: Wow, that's phenomenal. So, you made my day. You said MemSQL because just for our audiences, MemSQL was [00:15:00] the name of our company, Single Store, before it became Single Store.
And you also said Jack of All Trades. That's how I describe my job to everybody. I meet who don't know anything about technology. Speaking about the data itself, what were, you mentioned that use AI and ML models, I'm assuming even before the LLMs became very, very popular. What were these mostly used for?
Is it for engaging with the users in real time or was it to generate content or both? Yeah, it
Alex: was at that time mostly for, for example, building out recommendation systems. So we would take in essentially a history of all of the interactions that the student had with the videos and the content on our site and be able to glean from that.
where their strengths and weaknesses were. And based on that student profile that we were able to to build, we [00:16:00] could then recommend them a set of videos for them to really patch up the areas where they needed a little bit more help. This ties in kind of with that larger story of how we Try to build this end to end holistic learning journey that starts with really building knowledge and extends all the way to, you know, how do we actually make sure that in practice students really are able to solve and apply the skills that they're learning.
So it's a little bit of both worlds, yeah.
Madhukar: So I'm assuming because you know, you're being found directly by the students, you're mostly product led. Growth company. And so in that case, do users or students, they start off on a free trial and then go to a paid and do you think about that entire user journey as the user comes in, which in this case is a student.
I'm sure there is a different side as well for the educators and the creators. But typically, is that how you would describe yourself as a product led trial [00:17:00] to a premium feature platform.
Alex: That's exactly right. And during the trial process, we are doing deeper data analysis of what users are doing, trying to identify, again, where those weaknesses are and Be able to construct lesson plans to help them with their, with really deeply understanding the concepts that they're learning.
Madhukar: As you witness growth, like, what, what were some moments where you were like, Oh, wow, that just happened? And now we need to change our tactic, or now we need to go and scale, both on the technology side as well as your go to market. And what were some of the lessons you learned with the
Alex: growth? It kind of comes hand in hand with the needs of, like you mentioned, being a product led organization.
So when we were experiencing growth, the one thing that we realized was, you know, sitting students down and conducting these more, you know, qualitative, Feedback sessions with them, [00:18:00] getting them into focus group. That wasn't really all that scalable. Sure. It works really, really well, and it's still something that we do today.
As the amount of traffic that you get on the site has the number of users that begin interacting with our content, as that number starts to grow, there needs to be better ways for us to gain these deeper insights. And that's where we started the exploration of, you know, how do we best set a system up for ingesting all of the data that's being created by our users during their time interacting with our site and then afterwards is how do we actually create a team that can Both understand what that data and then be able to create actionable items off of that so that we can close that loop on that, you know, that PLG cycle and make sure that whatever lessons that we're being told by the students through data, that we're being able to integrate that into product.[00:19:00]
Madhukar: How do you manage that data, which is both the transactional, the user? The behavioral data with the content and the product usage data. Is that all part of your one single platform or the multiple different databases and you do a bunch of ETL
Alex: around? Exactly. Yeah. So we do a bunch of ETL. We have a couple different databases for this and it's again using most of what is offered out of it is where.
Big users of Redshift, we have S3 as our as basically our cold storage and, you know, we leave a lot of our logging there and from time to time we'll pull it out and see if, you know, did, was one of our hypotheses true when we kind of, and, and try to learn from from those, you know, whatever experiments that we're running and from there, that's when we start baking it back into the, the, the product side.
But we do, in terms of the infrastructure, have a few different databases and leverage a lot of the different technologies that are [00:20:00] offered.
Madhukar: And going forward, do you think that's how it will stay for you? Or do you think there might be a time where you might want to mix and match different kinds of data in real time?
Or do you think that that's just a pipe dream?
Alex: No, I say absolutely. I think as we start to As we continue to invest more and more into AI, particularly around generative AI and with LLMs, that there are new different types of databases, for example, using, you know, Vector Davis. And we do have some of that up, but really being able to take that data Do some transformation aspect onto it, put it into something that's a little bit more ready to, and just for ML and AI purposes, that is something that we do have on the, on the roadmap.
Madhukar: And, and speaking of, are you able to speak about some of the use cases of how you're either looking at using a large language model or if you're already using one?
Alex: Yeah, so we do use them in quite a few different [00:21:00] scenarios. I think that's one of the Most groundbreaking things of what's happened over these past year or so is really the leap from ML and the leap from AI being something that's typically happening behind the scenes to now being at the forefront of actual, what the actual product is and being much more user facing.
We do have a Like many of the other education apps out there, we do have a chatbot that users can interact with, they can ask questions with it, and it's really modeled after, again, a lot of the tutoring insights that we were able to glean from previous experiences in the past. Really, again, bringing some of those learnings into how do we best create AI tutors that can really help students get to the crux of where their problems are at.
There's also other applications of LLMs that we've been able to use. And one thing that's, that's been starting to become more and more apparent is all of the [00:22:00] different emergent capabilities of LLMs and us being able to use LLMs. Basically, yes, classification models in ways where, you know, before we might have been built a, you know, simple neural network, it's something that now LLMs are able to do.
Do you
Madhukar: think in the future, there may be educators that would be using the vision, audio, all of these AI capabilities to build videos that would look like humans, but they are far better educators than humans? and might be more popular, or do you think humans would still be in the loop?
Alex: I think right now, from all of the data that we're able to see, humans will, at least in the immediate future, still be very much a part of the human experience and the learning experience.
And I think the reason behind that is just because the learning experience is also inherently human. And you have to have some of that human element behind it to really [00:23:00] effectuate Great learning right now what we're seeing with Chatbot and I know that, you know, it's a popular tool among students these days, but a lot of the feedback that we're seeing from students is the fact that it's still very much like chatting with a chatbot and the ability to really build out, I think, deeper relationships, which really is, is at the heart of what great tutoring and great teaching is.
Those are still, I think, outside of the realm of what's what's possible.
Madhukar: So, I'm sure since you have been creating so many videos and a lot of students have been using those there must be some interesting facts about what is more popular, what works, and what doesn't work. Like, do you have some some high level trends that you can share or could you share?
What's the most popular video or topic? Like, do people, do [00:24:00] students like Advanced calc more or what are some of the interesting things that you see in the data that you could potentially talk about?
Alex: Yeah, absolutely. And I think you kind of hit the nail on the head right there is where we still see a lot of it.
This even goes back to day one when we had first launch Numerade is I think where students really struggle is making that leap from the high school curriculum over to the college curriculum and typically the first class that students have. when they set foot on college campuses is like an intro to calculus class.
And if you think about the curriculum that students need to transition off of into college, right? So going from something like pre calculus into a college level calculus course, there is fairly large walls that the students have to scale in terms of just being able to really set themselves up for success when they're tackling their first college level course.
[00:25:00] So that that's really the Those set of intro courses that are part of really any engineering and even most, most STEM majors, that's where we see still a lot of interaction with and a lot of engagement with.
Madhukar: In terms of STEM, do you, do you have some, I'm sure you do, some success stories where students reach out to you and say, you know, it was extremely tough for me to read through a textbook.
But now I'm certainly an expert in calculus or something like that. Do you have stories or do you have students who come and tell you that?
Alex: Absolutely. Any of the greatest stories are the ones that kind of tie in with our larger mission of really trying to Create more access to these great resources for students and the ones that really resonate with us the most are the stories from students who are coming from high schools or community colleges where they [00:26:00] don't have as much access to either Courses or even to support from their professors or their TAs for whatever reason and seeing the success that they're able to have to have Numerade come in and be able to be a partner in their learning journey to be able to offer them courses that they otherwise wouldn't have had access to and just see themselves really prepare for the next level of learning.
Those are the those are really the great opportunities. Stories that we've been able to hear from our students.
Madhukar: And now switching, switching gears into the application and the app developer side of things. If you were to redo this all over today in the world of LLMs. What would be some things that you would do differently while thinking about the application, the architecture?
Alex: Particularly around all of the new developments around LLMs [00:27:00] and the fact that pretty much every single week, and I think it's great timing just because on Monday OpenAI had their developer day, is the development cycles I think have really compressed a bit just because of how much new innovation is being done in the LLM space.
So I think for, maybe less tactically, but I think strategically in terms of how you approach up development is really around keeping in mind the fact that whatever that you're working on right now, something new, bigger and better is going to be deployed within, you know, the next month, the next quarter or so.
And that was going to absolutely change up some of what you're able to, to do. So I think maintaining that flexibility and, and, and being able to accommodate. All of the new innovations that are really setting up yourself for success to integrate a lot of these new capabilities is something that is, I think, really particular to the era that we're in right now.[00:28:00]
Madhukar: And what's your vision for Numeraded for the next two to five years in terms of, you know, all the advancements that we keep hearing about in open AI and. LLMs in general. What do you think is the next big thing for, for your platform? Yeah,
Alex: I think going back to the notion of humanness and how do we start really creating these experiences that go beyond simple, you know, chat based, back and forth.
Text messages. I think that's really where, for us, we're really starting to explore is how do we bring more of the humanity to these learning experiences than what is currently existing on the market. So, I think all that to say is there's a lot of opportunity that exists out there. I think it all starts with, you know, what do students want and by understanding where their pain points are and how they envision success to be in their own academic lives and [00:29:00] building for that.
And making sure that, you know, we're leveraging technology in the right ways is you know, how we want to really set ourselves up for the next two to five
Madhukar: years. Very good. So let's move to the last section which we call Quick Hits. And typically we just go through a round of quick three or four questions.
And the idea is to just answer. depending on what comes to your mind, so that if this ever gets harvested by an LLM in the future, they would know all the details about you. So first and foremost my question to you is, how do you explain your job to somebody who is not in technology?
Alex: I think our DevOps Lead has a really great metaphor for this and, you know, typically when he's talking about Any type of blocks of code or scripts that we're writing, he refers to them as incantations or as spells.
And so anytime we're kind of writing out a command, right, it's kind of like reciting a spell. So I think that's really [00:30:00] what we do on a day to day basis is pull things out of our spellbook. Recite them out loud to this mystery machine, call the computer, and then see what happens. So essentially we're, you know, like wizards in a way, but I think probably a little bit less exciting in many respects.
Madhukar: All you're missing is a hat, but then I could see you explaining yourself as a wizard. Second question, do you think chat GPT is sentient? I hope not,
Alex: just because there's some points where I get sometimes frustrated with its output, and I'd be Very remiss to know if it was, you know, sentient and would, you know, at some point, and I know that there's other platforms out there building actual, you know, humanoid and, and, and scaffolding for, for robots and knowing that maybe one day it will have a pair of legs.
chase after me. That's a little bit concerning. So hopefully it's not, hopefully it's not sentient.
Madhukar: Next question. What do you like to do outside of work? I have
Alex: a little four year old and [00:31:00] so he definitely keeps my hands full. You know, just give him a hot dog and he pretty much has the energy of a fusion reactor.
So chasing him around and making sure that he's not jumping on cars and basically keeping him out of the mayhem is my other full time job.
Madhukar: What's the most bizarre or most strange question you might have heard from anyone about technology from you, either from A user or even outside of your platform.
Alex: So sometimes we do have users who subscribe and sometimes they do have, you know, they have existing subscriptions with our other competitors and some of them will actually reach out to us and say, and ask us to cancel their subscriptions. on these other competitive platforms. And although I wish we had the power to do so, unfortunately, many times we aren't able to help them out there.
Madhukar: Yeah, I remember once when I used to work for HP, and [00:32:00] I told somebody when I was at a university about to speak on something, and they literally handed me their laptop and said, Can you fix it, please? Because it was in HP and it was broken. All right, last question. Simple yes or no? Do you think we are living in a simulation?
No, hopefully not. Great, awesome. So thank you so much, Alex, and this has been great. I love learning about what you are doing, and of course, how it's affecting students, and I love the fact that you and I think about the same thing, which is One of the most human thing that we can do is learning, and I always believe personally as well that humans would be in the loop when it comes to learning.
So thank you for doing what you're doing and appreciate you being on the show.
Alex: Thank you for having me and really glad to be able to share a little bit more about Numerade today.[00:33:00]