The Simplicity of Mathematics
A number of students recently discussed the fact that they understood nothing in a recent course they took from a high-status academic. The math, they said, was too difficult. Yet they were able to quickly grasp the same mathematical techniques when I went through them with them in a series of meetings.
It is not surprising that they could understand. Mathematics is simple. It is the rigorous formulation of well-defined relationships. We utilize the formal language of mathematics because it is simple. Mathematicians place great emphasis on simplicity: Theorems that survive and prosper are those that are simple and elegant. Yet, often this is not what students’ experience. Instead they often encounter mathematics that they feel or fear is beyond them.
Sometimes this is the students fault. To learn, you must expend effort. You must build your knowledge gradually, ensuring you have the bases required for further understanding. You must also not be intimidated by apparent complexity, and trust that with effort these complications will be shown to have a clear underlying explanation. But it is obvious that the difficulties encountered by students in machine learning is often not due to these factors. There is, I fear, something rotten in the state of mathematical education in machine learning.
Mathematical competence is very highly regarded in fields of applied mathematics, including machine learning. It can be held up as the separating distinction between the high-prestige theorist, and the lower-prestige practitioner. It can serve to elevate an academic over those who struggle to comprehend the same material. Unfortunately, in such a situation, people are inclined to over-emphasize the complexity of the mathematics they work with. Why simplify if what you are aiming to do with your presentations is to underline your ability to work with complex mathematics? Applied mathematicians can all too easily seek to signal expertise by complexity. This may be particularly true if they are not, in fact, particularly sure in their mathematical depth. We can perhaps regretfully accept this one-up-man-ship in the prestige chasing confines of academic conferences, it is unacceptable in the classroom.
However, even free of desires to look clever, it may be difficult for some theorists to teach the concepts they work with. Simplification requires a deeper understanding of the topics involved and this is not always present. Anyone can place formulas on slides. Most theorists can work with formula they are handed, and even contribute significantly to the development of the field. But understanding how these formula relate to the generally very simple ideas that led to them can be difficult. That is so because here, in the derivation of a robust theorem from a clever insight in the face of demands for rigor and numerous degenerate cases, we really can encounter complex mathematics. Understanding this can be difficult, and understanding it sufficiently well that you can explain this pathway clearly to non-experts is especially tough. It is often unnecessary for theoretical research. But it is this ability to see the simple ideas in the final formulations that students require.
Too many teachers of machine learning do not have the required knowledge to teach in this fashion. That, though, is not a significant problem – they can remedy this by effort, or seek to educate the same concepts in a less mathematical fashion. What is a problem is if they are unwilling to acknowledge this need or act on pursuing the remedy. But the same lack of confidence that leads to over-complicated papers can perhaps lead both to a desire to signal mathematical competence in their classes and to a denial of the need for improving their understanding of the mathematics involved in order to do this – lest this acknowledgment of their limitations dents either their peer’s conception of them, or their own self-conception, as highly mathematically competent.
Beyond bemoaning this situation, what can a student do? Firstly, they should understand what they can expect from a teacher. Assuming you have at least some mathematical experience, you should expect them to be able to explain to you any mathematics they are working with in ways that you can understand. But they can also consider the actions of a potential teacher. Those who teach machine learning should be happy to talk about the concepts in regular conversation. If you ask questions about how things might be different, they should not simply note that they are as they are. Almost always, the right answer to a question beginning “What if …” is not “No” but rather, “Of course, but…”. Perhaps they should also look beyond trappings of status. University careers are made on the basis of grant gaining. Success can be the result of many factors, some of which are either independent of scholarly expertise and teaching ability or positively opposed to it.
Good luck. And remember: Mathematics is both beautiful and simple. Anyone who tries to tell you otherwise either doesn’t understand the math they are working with, or is trying to sell you something.