Okay, so let me tell you about this Rocket Mortgage Leaderboard project I tackled. It was a wild ride, but super rewarding in the end.
Getting Started – The Initial Idea
First off, the idea was pretty simple: create a real-time leaderboard showing top performers at Rocket Mortgage. You know, sales guys, loan officers, the whole shebang. Management wanted to light a fire under everyone and boost competition, so a visible leaderboard seemed like a good way to do it. I thought, “Yeah, sounds doable!” Famous last words, right?
Digging into the Data
Next up, I had to figure out where the heck this performance data was coming from. Turns out, it was scattered across a bunch of different systems. SalesForce was the big one, but there were also some legacy databases and even a few spreadsheets floating around (shudder). So, first thing was, I had to build some ETL (Extract, Transform, Load) pipelines. Basically, I wrote a bunch of scripts to pull the data from these different sources, clean it up, and shove it into a central data warehouse.
- Extract: Wrote Python scripts using the SalesForce API and some custom SQL queries for the databases.
- Transform: Had to deal with inconsistent data formats, missing values, and all sorts of garbage. Used Pandas in Python to clean and massage the data.
- Load: Pushed the cleaned data into a PostgreSQL database on AWS.
That part alone took me a solid week. Data cleaning is always way more time-consuming than you think it’s gonna be.
Building the Backend
Now that I had the data in one place, I needed to build a backend API to serve it up to the leaderboard. I went with * and * because I’m pretty comfortable with Javascript, and it’s quick to get something up and running. I also used * for real-time updates. The leaderboard needed to refresh every few seconds to keep up with the latest sales numbers.
Here’s a rough breakdown:
- API Endpoints: Created endpoints to fetch the leaderboard data, filter by team, date range, etc.
- Database Queries: Wrote efficient SQL queries to get the data from PostgreSQL. Used * as a query builder to avoid writing raw SQL.
- Set up * to push updates to the frontend whenever the data in the database changed. This was crucial for the real-time aspect.
I spent a good chunk of time optimizing the database queries and caching the results to keep the API responsive. No one wants a laggy leaderboard.
Designing the Frontend
For the frontend, I kept it simple and used *. I wanted something that was easy to maintain and update. I also used a charting library (*) to visualize the data in a more engaging way. No boring tables here!
The main features were:
- Leaderboard Table: A sortable table showing the top performers, their rankings, and key metrics (loan volume, number of closed deals, etc.).
- Charts: Line charts showing trends over time, bar charts comparing performance across teams, etc.
- Filters: Users could filter the leaderboard by team, date range, and other criteria.
- Real-time Updates: The leaderboard updated automatically whenever new data came in via *.
I spent a lot of time tweaking the UI to make it visually appealing and easy to understand. First impressions matter, especially when you’re trying to motivate people.
Deployment and Testing
Once I had everything working locally, it was time to deploy it to the cloud. I used AWS again. I deployed the backend to an EC2 instance and the frontend to an S3 bucket with CloudFront for content delivery.
Of course, no project is complete without testing. I did a bunch of manual testing to make sure everything was working as expected. I also set up some basic monitoring tools to track the performance of the API and the database.
The Grand Reveal
Finally, the day came to launch the leaderboard. I was nervous as hell. But, to my surprise, it went pretty smoothly. People actually loved it! Sales went up, and there was a noticeable increase in friendly competition. Management was thrilled.
Lessons Learned
This project taught me a lot. Here are a few key takeaways:
- Data is king: Cleaning and validating data is crucial. Garbage in, garbage out.
- Real-time is hard: Getting real-time updates to work smoothly requires careful planning and optimization.
- Simplicity is key: Don’t overcomplicate things. A simple, well-designed solution is often better than a fancy, bloated one.
Overall, the Rocket Mortgage Leaderboard was a challenging but incredibly rewarding project. It was a great opportunity to learn new skills and make a real impact on the business. Plus, it was pretty cool to see my work being used by so many people every day.
