- Details
- Category: Blog
- By Stuart Mathews
- Hits: 2796
Since Itchy legs and tracksuit bottoms, I've been keeping to my exercise routine. I sometimes alter at which times I go. Recently, as it's getting colder and the mornings darker, I've opted to go for a run after work, sometimes I go during the day. Most of my work keeps me focused on my laptop, and coding takes a large portion of my time however this is what I prefer so I'm very lucky to be able to do it every day.
I've been thinking recently about abstractions, generally.
For example, an abstraction is a mechanism that reduces the details of something such that it appears simpler and less complex. I've been thinking specifically in terms of the reduction of complexity in software as the introduction of abstractions to simpler components, which when you collectively interpret all the resulting abstractions, it yields the same result as what the complexity that came before it did, but now it's just represented as a composition of simpler abstractions.
This is specifically relevant to design patterns as they are a cognitive abstraction themselves of a design idea, which has been reduced into its bare essentials without any verbosity or detail about anything else that doesn't affect the idea itself.
The problem with abstractions is that if you need an idea of what they are, or why they are present, it's difficult or nearly impossible to determine without details that are now missing from the abstraction itself. Details that would tell you about what the abstraction represents are gone which is counterproductive.
So in this case, you might need the complexity it explain the abstraction and that is the paradox itself. In this situation, you must know something about the abstraction first to appreciate or understand why the abstraction exists. So in this way, prior knowledge is essential and so abstraction is useless or very difficult to use/appreciate/explain if it is not recognised first.
And when you think about design patterns, research has shown that inexperienced developers cannot fully appreciate design patterns if they have not come access to them before or they are not easily identifiable.
This brings me to my main topic: teaching kids using abstractions.
For example, someone somewhere down the line said to me that the derivative is the rate of change. Someone else later taught me and said to me that it is the tangent to the curve. Someone else said it's the velocity, and that it's the rate of change of distance. Then, it was the change in y over the change in x or the ratio of the change in the function.... You can't teach like this, but sadly this is how we are taught.
The problem is all these terms are forms of abstractions, and they are all subtly different and thus they are not accurate learning representations, so if you use them at different times in the learning process, it's difficult to accurately explain the detail they represent because the different abstractions are somewhat confusing too, and you need the details when you teach.
Teaching should pretty much always remind us what the details or essence of the abstractions are...otherwise we'll forget and end up thinking in abstractions and never fully realizing what those abstractions actually represent...we'll just know how those abstractions work or are defined - which is not the idea, the idea is to represent the detail simply...but now, you don't know the detail that makes up the abstraction and you're just playing with abstractions now... You've lost your way.
It makes me wonder however if the pursuit of detail is actually a personal journey, where it's up to you to figure out what the details are from abstractions that are thrown at you? Should the kid be saying to himself, wait I don't understand this, I need more details and figure out the details? This is how it ultimately ends up though I think - you either decide you want to know the details or you decide you don't. If you don't (or are not sure if you know what you want), you'll get stuck in the myriad of abstractions and formulas and memorization techniques to get you by through school. And by you must get, because as a society we've mandated that you deal with this education we want you to have. I wonder if that's fair or what's better?
If you aren't exposed to abstractions, you won't be in a position to be confused by them and determine if you want to figure them out. So you potentially need this conflict, and perhaps that's why we have what we have in the education system. Kids being taught math/science/computers - some kids get it, some kids don't.
I'm not sure if it's the kids' fault for not wanting to know the details of the abstractions but there shouldn't be anything wrong with not wanting to know something, there is nothing wrong with not being interested in something, but ultimately they are actually punished for not being interested in the things that our education system demands. They will ultimately become stressed out in exams, learn stuff they don't want to learn or are not interested in learning, they'll get poor grades and comparisons of intelligence will be made - all because they were not interested in some particular area of learning.
Anyway, this system of needing to become interested...or else, is what causes a lot of abstractions being used to simplify things so that they can cope with this uninteresting thing they have no choice in learning, or can be easily dealt with if you're not interested in the details. Ultimately you're either in the camp of you became interested, you figured out the details or you endeavoured to understand it or... you never did and are now stuck in the system that demands from you what you don't want to give, or know how to give.
I'm not sure what the solution is.
Coming back to the derivative example, I bet many people aren't told why calculus was invented and what problems it solves now that could not be solved before it was invented. If people were told that before calculus, it was impossible to draw a tangent to an arbitrary curve (other than a circle and a few other shapes) and that calculus lets you determine the target line to any curve anywhere and anyhow...that might make a student more interested in why it was impossible in the first place, and why you might want to actually do that.
Secondly, if people were told that it can determine the exact measure of the change in a constantly changing thing at any time while it's changing, they might think wow that's pretty cool if it can be done (because it sounds bizarrely difficult).
Thirdly, if they were told that before calculus you could not measure the area of a space that is bounded by curves, like a squiggly square or giraffe shaped cookie cutter but you now can with calculus, perhaps they've think ok maybe I'll determine why this is useful.
I was taught by using abstraction, I don't think it helped me, but equally, I don't think I knew what I wanted to know, detail or otherwise and I think I just joined the system.
That's why I wonder if it's a personal journey to figure out the details, whether is a dedicated choice to set out and conquer the detail.
In a way I think it is because without that choice I would not have come to the detail:
The derivative is a measure of an angle that a tangent line will make with a curve, where the curve represents the constantly changing thing, and that the point on the curve that the tangent line hits, is the exact time that the constantly changing thing changes to a particular value, and that the angle measurement (ie the derivative) accurately measures the change to that value in comparison to changes to other values at other times. So now you can compare changes. The rate of change can be measured as this angle measurement.
Interesting.
- Details
- Category: Blog
- By Stuart Mathews
- Hits: 2356
These are a collection of the mind maps I've put together over time for certain projects. I find them quite interesting to review from time to time...
{gallery}mindmaps{/gallery}
- Details
- Category: Blog
- By Stuart Mathews
- Hits: 13682
Since Implementing a Vignette, I've been pretty involved in getting various machine learning algorithms to, well learn...
I've been playing around with SIFT feature extractors, Histogram of gradients (HOG) and convolutional neural networks (CNNs) and at times it's been quite interesting.
When I started learning about machine learning techniques, I wouldn't say machine learning was of immediate interest to me. I think ever since I did a course on Data management and Analysis, I kinda thought designing software was more my thing. Sure the graphs were cool though, and I like graphs but I think too much data fiddling well just becomes too much data fiddling. That said, I did not learn machine learning and the closest I got to classification was K-nearest neighbour clustering techniques.
With machine learning, the mathematics however is quite interesting, especially the partial derivative calculations that help you determine the impact that model weights are having on the loss function of your model (backpropagation). I did have to write them out by hand initially because otherwise I just would not understand it.
After you understand this, you start to understand that parts of machine learning are very much a brute force, nudge-it-until-its-correct sort of discipline - which is effective but this is an over-simplification of course, and there are more smarts involved.
What is really quite impressive is that pytorch has a built-in Tensor type that will track how each tensor's value impacts an expression that involves that tensor - and calculating the impact of the tensor on say loss function is just a matter of calling backward() and the entire object hierarchy involved in the expression is evaluated for its impact on the expression. Quite cool. This helps not having to worry about trying to calculate the chain rule manually on a piece of paper! Also, found it pretty cool how easy it was to move Tensors to the GPU to speed up training times.
With this, I've pretty much swapped Ruby for Python which has become almost an extension of me lately (same goes for C++ but more on that later). Interestingly while designing my feature pipeline, I found that python has no concept of private members and the convention is just to use double underscore in front of the method name. Abstract classes exist which was useful, as I designed a pipeline (which is basically this) that is based on interfaces that allow uniform interaction but allowing varying underlying implementation details.
My pipeline currently consists of 2 classifiers (SVM - Support Vector Machines and MLP - Multi-Layer Perceptron) and 2 feature extractors (SIFT and HOG) and one convolutional neural network (CNN) based on MobileNetV2.
The CNN I originally designed from scratch needed too much training than I had time to do and so the learning rate was poor. So I've been fine-tuning this one using pre-trained weights and I've just adapted it to learn the classes that I'm interested in.
I will say that I particularly enjoyed the learning around machine learning theory, for example understanding why a non-linear function (Sigmoid or ReLu for example) is used after linearly combining the input values, which is to produce a variance in the shape of the function which contributes to determining a function that best describes the input.
To this extent, this book was particularly useful in understanding 'why' and not just doing it and moving on which is so often the case with technical theory - not that this book was technical it was more practical and provided diagrams like this one - which my brain seems to appreciate. I wouldn't say I'm proficient however but I'm interested which was more than I could say before.
Pity I don't have an NVidia graphics card so I've been having to use the GPU in google colab and I hit the limit a few times while training but eventually got 83% validation accuracy which is pretty good.
Apart from that I've also been writing a lot of C++/OpenGL and finished a demo racing game, which I very much enjoyed programming. Its very simple but shows various important 3D graphical elements.
I've implemented an exponential fog effect and my scene is basically themed on Jurrasic park-styled atmosphere.
I've incorporated some meshes for the player car, the forest and the track. The path through the scene is calculated using catmul rom splines and the rear-view mirror is programed using a FrameBuffer object.
Most of the shader code is for the lighting effects and the fog. For the lighting, the Phong-Blinn model is used. Its been very interesting managing the vertex buffers and drawing 3D primitives etc.
What I'd like to do next is incorporate the library of code that I developed for the demo into a more abstract utility that I can use in the next thing I do.
In terms of Information Security, I wrote a critical review about securing software development processes as recommended by Clause 14 in ISO 27001 around the utility of implementing E-SecSDM, an engineering-focused software development model to improve the security of engineering systems that incorporate software in their design.
I think the learning I did on digital forensics, criminal law and network security previously was a bit more exciting than learning about ISO 27001/2 but like most things, its useful to know a little more than you did before so in this way, its useful.
I've been out and about running also - that knee niggle has seemed to have sorted itself out (well, I did implement a no-run policy for about 2 months) however my fitness has dropped off but that's ok - I've been slowly working my way back up. My last couple of runs were slow but they were pretty nice especially now as the sun is starting to come out.
Speaking of which, maybe I should go for a run. now....
More Articles …
- Design Patterns: Representation, Transmission and Dependencies
- Sufficiently Complex
- Deadlocks and databases
- Enterprise System Integration
- In the meanwhile
- Try Monad, Progress and Lockdown Rules
- Agile and Model Driven Development
- Rails, Euclid and Generating Mazes
- Set Theory, Ruby and Upgrades
- Time shifting, CS algorithms and Game Architecture
Page 16 of 182