I know it’s been a while since my last post here. A lot of water flew during that time, lots of things happened. One of them was my new pet project, which I am going to share with you.
It happens that my younger son is starting secondary school next year. Since in our local area there is a choice of few governemt, Catholic and private schools I wanted to find a way to compare schools performance.
Sure, there is a lot of information readily available on the Internet. There is an excellent Better Education website (it recently improved a lot on the UI side of things 😉 ). Also, some raw data (albeit in the PDF format) is available on the Victorian Curriculum and Assessment Authority website. However, I wanted a bit more functionality, a DIY approach, and to have a fun in the process.
So here were my requirements:
- To create and implement mathematical model for the school ranking – based on the historical VCE data.
- To be able to sort schools based on ranking score, school type (government vs catholic vs private).
- To be able to search and filter schools based on names, geographical locations.
- To be able to visualise data (i.e. on the maps)
- Microservices approach
- Develop on Mac, deploy on Linux
- Preferably do not use Ruby, Rails or R – learn something new instead
Here is the rough high level diagram of the setup:
So, as you see, it is all very simple. We will have a database with, well, our data. The REATful JSON API Service will present data to the UI component, which will display it in the browser.
To populate database we will use an ETL (aka Data Loader) service.
In regards to the implementation particulars I am going to use:
- Database – MongoDB
- API and Data Loader service – Golang (hence the title of this post) with MGO and net/http packages.
- UI – AngularJS
- Data Service – the abovementioned VCAA reports as well as datasets available from www.data.vic.gov.au.
- PDF to CSV conversion – Tabula.