dbt Core™ is an open-source tool by dbt Labs that enables the application of software engineering principles to data analytics. Simply put, it allows you to write, test, document and version control your SQL code in a single repository. No more fragmented, undocumented and untested SQL code (views) in your database!
So what are the tangible benefits that the adoption of dbt can provide for your team?
First and foremost, dbt provides a standardized way of organizing and testing your analytical SQL models in a single source-of-truth repository. This is especially important if your data warehouse has grown into a spaghetti mess of untested SQL views. Having tests defined for your analytical models and being able to run the tests on-demand will give your team the confidence to make changes to existing models and deploy them to production. This in turn increases your team’s productivity.
Second, you can automate tests in dbt Cloud, CI/CD, Airflow or any other pipeline orchestrator and get notified when your models break (e.g., due to source data changes). This gives you a head start on fixing the defects before they ever reach your end users.
Third, you can generate and publish a documentation site for your data warehouse using dbt’s built-in “dbt docs” feature. The documentation site allows anyone in your organization to explore the analytical SQL models and their relationships in your data warehouse. This is a step towards having documented and curated data sets available for anyone in your organization to experiment with.
I’m a dbt Preferred Consulting Provider and I provide dbt deployment, development and training (English and Finnish) services for organizations that have an existing data warehouse or other relational SQL storage, or do not yet have a data warehouse but are planning on setting one up.
Being able to adopt dbt in your organization doesn’t require “hardcore” data engineering skills. I’ve trained dbt to a wide range of audiences ranging from product development and mechanical engineers to data analysts and scientists. As a prerequisite for the training you only need to know SQL and some basics of git version control. And if those are not in your toolbox yet, we can cover them during the training.
Data engineers are the backend engineers of the data world. We take care of the data infrastructure so that data analysts and scientists can focus on what they do best.
I provide data infrastructure review, design and implementation services. In development work I adhere to software engineering and DevOps best practices, such as automated (end-to-end) testing, CI/CD and Infra-as-Code. I’m also a proponent of the Test-Driven-Development (TDD) methodology.
Analytics engineer is an emerging role in the data world. The role is similar to that of a data engineer but it leans more towards data analytics, as the name might suggest. In other words, analytics engineers provide clean data to data analysts using automated processes.
I can help your organization with your analytics needs ranging from simple scripts to more scalable and automated solutions. For more information on my analytics engineering portfolio, see dbt deployment and training.
We all have our strengths – and that’s a good thing! But sometimes you need specific skills and experience to take things to the next level. Let me help your team get there.
Kind words from clients.
“We used to have most of our analytics in Excels which eventually became a bottleneck for our product development. Simo helped us automate these processes by designing and implementing a data infrastructure tailor-made for our needs. Furthermore, he trained our product development team with no previous data processing experience so that they could understand and independently make use of our new analytics capabilities. Now we can handle large amounts of data in an automated manner which has accelerated our product development efforts and improved our decision making. We are really happy with the services of data&mie and will definitely continue our collaboration!”
International Advanced Technologies at Globus Medical