For better or worse, data engineering’s duty in 2021 will be to scale beyond the scope. As a result, several definitions of the position have emerged. Is the data engineer responsible for more analytics (as defined by the new role description of analytics engineer), data pipelines, infrastructure management (DevOps), or machine learning engineering? Basically, it’s becoming hazy as to how a typical data engineer will spend his time. These categories, however, come under technical tasks, and we typically overlook the fact that they only account for a small portion of our time.
Dennis Hume, a data engineer at alcohol e-commerce site Drizly, for example, is now in charge of moving the company’s data orchestration to Dragster. That project lays out what Hume considers to be a data engineer’s role.
“At the end of the day, data engineering’s purpose is to help data stakeholders across the company, not to be a bottleneck for data work,” said Hume.
MORSE Corp., a software development business, follows a similar approach to cross-team cooperation. Lena Bartell, Lead Data Scientist, is in charge of a team of data scientists, software engineers, and data engineers that are continually collaborating to design algorithms and data processors.
Built In spoke with six data engineers to get a better sense of what they do on a daily basis and how they are supported in their professions.
What is the most crucial (hard or soft) skill for a data engineer to be successful in their position?
One aspect of MORSE’s responsibilities that may surprise people is that data engineers (or any other field) are not segregated into their own group. Engineers with a variety of talents are mixed into teams, so you’ll constantly be working with folks from both comparable and dissimilar backgrounds. This is beneficial in terms of assembling a diversified and capable project team, as well as allowing us to learn about new issues.
What is one aspect of your job as a data engineer at your organization that would surprise people?
You’ll need a wide skill set to be a good data engineer. Programming, SQL, cloud, data and software engineering, design, and data science are all examples of this. In addition, an engineer must have a thorough understanding of the company and the ability to successfully interact with stakeholders. Everything revolves on data. It is critical to be inquisitive and educated about cutting-edge technology. The solution is to love (data)!
