Chief Quant, Director of Analytics at Math Academy. Posts about learning, upskilling, math education, Math Academy, and more generally, stages 2-3 of Bloom's talent development process in hierarchical skill domains.
New on JustinMath.com - How To Explain Anything; Solve Today's Problems Today; How To Get Eyes On Your Writing
|
Hey! You're receiving this email because you requested to be notified about new posts at JustinMath.com. Here are 3 new posts:
And a new podcast: American Enterprise Institute’s “The Report Card” Podcast: Math Academy
You know the kind of trainer an actor gets to prep them for the superhero role in a Marvel film? We're that, for math. If you want to understand what we’re doing but don’t have time to skim our 400+ page book, this episode sums it up in about an hour.
Here's a summary of what we covered:
[1:34] The big problem in math education is a lack of individualized instruction. In a classroom with one teacher teaching the same thing to all the students, it's way too easy for the top half of the class and way too hard for the bottom class. What we do is pinpoint the exact problem that each student should be working on right at this moment to make maximum progress in their math learning.
[4:46] So much difficulty in math learning can be traced back to missing prerequisite knowledge. That's why it's important to start each student off with a diagnostic that combs through many years of prerequisite knowledge that they need to know to succeed in their chosen course. If we find any knowledge gaps, we fill them in before asking the student to learn any more advanced material that depends on it.
[6:50] We get a very high-resolution picture of the student's knowledge profile by overlaying every question/answer event onto a structure called a "knowledge graph". The knowledge graph encodes all the dependency relationships between mathematical topics. We leverage it to squeeze a ton of information out of every single question that we ask the student -- not just figuring out what they know and don't know, but also figuring out exactly what learning tasks they should be working on to maximize their learning efficiency every step of the way.
[8:44] Elsewhere, lots of students struggle with calculus due to gaps in prerequisite knowledge. Good teachers know this, and try to fill those gaps, but there's a limit to how well the teacher can do this because all the students have knowledge gaps in different places and the teacher can only teach one thing at a time to all the students. But we can target these gaps precisely, backfill them, and move on based on what each individual student knows -- fully individualized instruction for all students in parallel, delivering exactly what they need to work on, focusing on those weaknesses and not wasting time on things they already know cold.
[12:05] If you have more talent/aptitude, then you're going to get more bang for your buck out of practice. You're going to require fewer reps before getting solid enough to move on, and you're going to generalize more naturally. However, of the students who get all the way up to calculus before struggle really sets in, the biggest roadblock is typically not talent/aptitude but rather gaps/weaknesses in prerequisite knowledge, an issue that can be resolved with fully individualized instruction.
[14:50] Math Academy origin story: Jason & Sandy coached their son's 4th grade math field day team. That turned into a pull-out class 3 days per week the following year. The superintendent came by and was shocked to see 5th graders doing trigonometry, advanced algebra, even a little bit of calculus. So he asked Jason & Sandy to create a pilot school program in the Pasadena Unified School District.
In the program, which came to be called Math Academy, students learned all of high school math (prealgebra through precalculus) during 6th/7th grade and took the AP Calculus BC exam in 8th. Students were invited to the program by scoring in the top 7-8% of a middle school math placement that all students in the district took at the end of 5th grade. Keep in mind that about two-thirds of students in the district were on free or reduced lunch, and also, nearly half of Pasadena K-12 students are educated in private schools, compared to the California average of ~10%. In other words, generally speaking, these were not smartest kids in California, and their parents were not Caltech professors.
Jason developed software to automate the process of assigning/grading homework, and together during the pandemic we upgraded it to figure out what each individual student should work on and teach it to them directly without any human intervention. We worked like maniacs to get it ready before school went fully remote the next year, and once we put the school program on it, educational outcomes (including AP Calculus BC scores) skyrocketed. Because of the software, our students experienced a massive learning GAIN, not a loss, during the pandemic. Naturally, it only made sense to keep the school program using the software even after class returned in-person.
[21:55] We have spent thousands and thousands of hours over the years building and fine-tuning our knowledge graph. It's not off-the-shelf, it's not automatically generated. It's the hard work from domain experts, primarily our director of content Alex Smith for the forwards graph (what are all the prerequisites you need to learn in order to unlock a topic) and myself for the backwards graph (when you practice a topic, what component sub-skills are implicitly getting practiced and to what extent).
[25:04] We analyze our knowledge graph by overlaying a big heatmap of where students are doing well or struggling at various parts in the graph. It's almost like traffic intersections in a city -- which ones are where most accidents happen? Let's go make those safer. We've been building and refining the knowledge graph for nearly a decade now with all these analytics.
[27:22] We have a wide variety of user segments. We can help anyone who seriously wants to learn math. Basically, anyone in any sort of educational situation, kids, adults, public school, private school, charter school, homeschool, grade school, high school, college, students who are accelerating, students who are just trying to keep up, "math team" people, people who don't yet think of themselves as "math people", adults changing careers to a math-ier field or pursuing a math-ier subfield within their current career, the list goes on and on.
[29:31] The best predictor of how long someone will use the system and how much math they'll learn is what kind of habit structure they have in place. Students who are consistent, as opposed to sporadic, go much further. It's that simple.
[34:13] The only math learner persona we can't help is the crammer -- the student who has an exam in a week, is nowhere near prepared, and wants a "quick fix". We are like a gym, and there's always people who walk in the gym and think they're going to work out really hard for a week and look like Thor by the weekend. There is no way to make that happen in a week, no matter how hard you work out. If you show up consistently, like 3-6 times per week for a 30-90 minute session, and then you keep that up for months, then you're going to come out looking like the mathematical equivalent of a Greek god. But if you are looking for some kind of easy, "how can I change my life in one week," then I'm sorry, I don't know what to tell you.
[37:16] We alternate between minimum effective doses of text-based guided instruction followed by active problem-solving. It's the mathematical equivalent of a tennis instructor showing a quick demonstration of how to hit a ball, just for a minute, and then students practice hitting the ball with that technique until they're solid enough to move onto the next technique.
[40:16] Real-time reactions and hot takes: Jason on collaborating with school districts, my thoughts on the edtech industry, Jason founding a company with his wife, my experience interacting/growing on X, Jason's impression of Waymo, my impression of math textbooks, Jason's thoughts on the "move fast and break things" ethos, Justin's thoughts on people's screen time concerns.
[52:10] People say, "just give me the intuition." But intuition comes through repetition. That's how you get the automaticity, the natural feel, and that's what intuition is.
At the same time, it's important to be efficient. Don't work 100 problems of the same type in one day. Maybe do 10 to start, then 5 the next day, another 5 a week later, and so on, while you're filling the empty space with practice on a ton of other skills. You have to get your reps, but you also have to distribute them out over time. That's how you learn efficiently and build long-term retention.
When people want their math learning to be less skill-heavy and more concept-oriented, what they're often really saying is that they want a fast overview of a subject without going into the details, without really getting your reps on everything. A video that explains all of calculus in an hour, or how neural networks work in 20 minutes.
But what we're focused on is building up a true level of mastery. Not surface-level, not shallow. The optimization problem we're solving is NOT "how fast can we imbue you with a shallow level of understanding, enough that you can tell your friend something cool or that you think you have opinions about it." What we're focused on is how quickly we can get you to operating mathematically almost like a professional musician plays their instrument, or a professional athlete plays their sport.
[55:51] As a rule of thumb, if it wouldn't work in sports, it's not going to work in math.
[57:31] Students on our system typically learn about 3-4x as fast as a normal class. That's why, in our school program, the students could go from pre-algebra through AP Calculus BC in 3 years, from 6th-8th grade. When that first happened, and the Washington Post wrote articles about it, lots of people couldn't believe it. Which is why we had them take the AP Calculus BC exam so we actually have results.
[58:49] We hear all the time about students who are behind in their school class, and then use our system to catch up, and then start crushing their class, and then go well beyond their school class -- as well as the resulting change in the student's level of confidence. In just one year or less, just months, a student can go from thinking "I'm not a math person, I'll never be good at it" to "I'm crushing my school class, it's so easy." That change in the student's experience does wonders for their confidence.
[1:01:28] Is there an upper limit to how much math you can do per day and have it carry over into real learning results? Think about it like going to the gym. If you work out for 45 minutes, 5-6 days per week, you'll get in incredible shape. You can do more if you want, but there is a point where you hit diminishing returns. Whether it's Math Academy or the gym, it really comes down to how long you can sustain a productive full-intensity effort. It's hard to keep that up for multiple hours, though you might be able to get better mileage by splitting up a multi-hour session into a shorter morning session and evening session. But every person is kind of different in their breaking point, how much they can stay focused and work intensely on the system. In general, one hour per weekday is what we've found to be the upper end of a sustainable approach for most students.
[1:03:38] We make students do review problems indefinitely into the future, but with expanding intervals -- spaced repetition. It's the optimal way to keep your knowledge base fresh enough to keep building on it without constantly having to go back and re-learn things. But we make this review process as efficient as possible by tracking all the subskills that are implicitly reviewed when you do a review problem, and we're always trying to select tasks that kill many birds with one stone by exercising many subskills in need of review.
[1:08:41] Lots of people mistakenly think that students need a million different explanations of the same thing, and that one of those explanations is going to stick, and it's different for each student. But really, all you need is one really good explanation that's been battle-tested across a large number of students, and the students need to come into that explanation with all their prerequisite knowledge in place.
If you do that then you can get students learning the skills really well -- students pass our lessons over 95% of the time on the first attempt, and over 99% of the time within two attempts, without any additional remediation (because enough knowledge has consolidated into their brain from the first attempt that it makes it cognitively easier for them to get over the hump the second time around).
That's often surprising to people who think that every student needs a different explanation, but typically what they're seeing is a symptom of the student lacking prerequisite knowledge, and you're trying to come up with some explanation that allows them to grasp "enough" of the topic (not the whole thing) while at the same time not requiring too much in the way of prerequisite knowledge they're missing.
[1:11:08] What makes math hard is the same thing that makes climbing a mountain hard: the steepness of the gradient. What we do is break every steep section of math into smaller steps. If you break things into small enough steps, anyone can learn. And that's what we do with our analytics: where are the congestion points? Where are students struggling? It's always where we're trying to do too much at one time, so we break it up into more steps.
[1:12:25] It's so important to have a reliable source of truth about what a student really knows, and grades are no longer a good source of truth. You remove test scores from the admissions process, the last objective metric and the last Jenga block, and you get bad situations like at UCSD where 8% of students were not proficient in middle school math. So many issues in education stem from a student having a piece of paper that says they've learned something when they actually haven't.
Best, |
Justin Skycak
Chief Quant, Director of Analytics at Math Academy. Posts about learning, upskilling, math education, Math Academy, and more generally, stages 2-3 of Bloom's talent development process in hierarchical skill domains.
Read more from Justin Skycak
Hey! You're receiving this email because you requested to be notified about new posts at JustinMath.com.(If you've changed your mind, click here to unsubscribe, please avoid marking spam.) Here are 3 new posts: The Most Mathematically Gifted Student I Ever Worked With Still Needed To Be Pushed to Learn Calculus~200 words • Even when you’re doing what you love, there will be grindy phases. But kids typically don’t understand this. It’s often up to parents, who can see the long game, to push...
Hey! You're receiving this email because you requested to be notified about new posts at JustinMath.com.(If you've changed your mind, click here to unsubscribe, please avoid marking spam.) Here are 3 new posts: How I Went from 19 to 25000 Followers on X in 18 Months~1050 words • Here's the progression I followed to level up my writing and build an audience. It’s reproducible if you're willing to put in the work. The Cycle of AI~150 words • Progress is made in AI, people lose their shit...
Hey! You're receiving this email because you requested to be notified about new posts at JustinMath.com.(If you've changed your mind, click here to unsubscribe, please avoid marking spam.) Here are 3 new posts: Why Can’t College Students Do Middle School Math?~1800 words • 1 in 12 incoming UCSD freshmen don’t know middle school math, and the remedial math course was too advanced, so UCSD had to create a remedial remedial math course covering elementary and middle school math, and a quarter of...