EdExplorer

Ingest demographic, financial, and performance data for Connecticut schools and districts and visualize it using graphs, allowing the user to compare data with ease.

Create a progressive web application allowing visitors to add the app to their home screen.*

* This was not an original project goal, but it was a key request from the beta testers. Read more on this in the Highlights section.

Allow visitors to print the data to a PDF file.

Data Availability: There were two key challenges with the availability of data.

1. Not all data was released at the same time, with financial data often unavailable until a full year after the end of the school year. This meant we could be showing 2022/2023 data for performance metrics, but 2021/2022 data for financial metrics, on the same page.
2. Not all Districts or Schools had the same amount of data, nor did we always have complete data sets for a full year. This meant when comparing districts or schools, there could be instances where one entity would have data for performance, and the other would not.

The second major challenge was importing the provided data from the SSFP Data Team.

The third and final major challenge was giving the ability to allow users to download district/school data in a presentable format i.e., PDF.

The application needs to be translated into 4 different languages.

We created a versioning system to help tackle the primary goal and numerous challenges.

The client wanted to display the most recent year of data. To accomplish this, we needed to review all imported data, extract the latest information, and then present that on the front-end charts and graphs. However, this was a resource-heavy process and wasn't prudent for every request. To alleviate the load on the server, we developed a database structure to store both raw imported data and presentable frontend data. 

This allowed us to streamline data retrieval and optimize performance, automatically updating frontend data whenever new information was imported, ensuring that users always had access to the latest insights.

The client provided raw CSV files, prompting collaboration with the data team to establish a mutually compatible format. After importing the data, we combined crucial information to generate unique keys for each data row. This facilitated the uploading of slightly modified data, ensuring that any changes would overwrite existing entries without duplication.

Creation of the database structure for raw and presentable frontend data enabled the implementation of a previewable URL without impacting production data during imports. This approach ensured seamless data deployment with zero downtime.

In our UX design, we prioritized the visibility and clarity of the latest dataset. It was crucial for users to easily identify the specific year associated with each metric. We also implemented this feature in a subtle and non-intrusive manner, enhancing the user experience while delivering key insights.

We collaborated with the client to identify the key data points for inclusion in the PDF downloads. We then crafted a meticulously formatted document that aligns seamlessly with their brand aesthetics.

Enabling data downloads posed challenges, especially considering the inclusion of data charts. To address this, we developed an HTML template to dynamically generate printable PDFs, incorporating data retrieved from the database. Leveraging the extensibility of Chart.js, we produced images of specific charts for integration into the PDFs.

Recognizing the resource-intensive nature of this process, we implemented the following steps:

• Upon initial request for a specific district or school, we generate the PDF.
• We dynamically generate graph images and gather data.
• The resulting PDF is saved on the server.
• Users are then redirected to access the PDF.

By following this approach, subsequent requests are instantaneous, ensuring optimal server performance without overload.

To accommodate the application's translation into four languages, the downloadable PDFs also required translation. To handle this, we implemented a system to detect the user's language and generate PDFs accordingly. However, this solution introduced an additional challenge, unrelated to our initial design

As we were creating static PDF files, representing a "snapshot" of data at a specific time, this meant that once the data was updated, previously generated PDFs became obsolete.

To address this, we utilized the Versioning System:

• Each new version received a unique identifier.
• We prefixed the PDF file with the unique identifier
• When a PDF for a district/school was requested, we checked if the current version of the PDF existed, and if it did, we provided it; otherwise, we generated a new one.

This approach ensured that all download requests provided users with the most up-to-date data available.