As Product and Technology teams continue to specialize, the past 10 years have brought out the emergence of dedicated Data and Data Product organizations. With this new specialization brings new viewpoints to center of stagnant companies, but introduces new challenges across Technology and Product teams. A similar pattern could be observed in the evolution of Product and its birth out of Technology. In the long run, this resulted in a lot of gains for the Technology field as a whole. E.g., a focus on iterative problem solving, viability/initiative risk management, and user centric observations/solutions. What I am not describing is the bureaucracy of agile religion that plagues Product. A similar concern on the data side, everything needing to be solved by Artificial Intelligence/Machine Learning (AI/ML).
With this new birth of focus for companies comes new communication challenges we haven't seen before in Technology. For the most part, Product teams were symbiotically attached to Technologists. Even the worst of communicating cross functional teams were forced by design to exchange information. In comparison to Data and Technology, Data organizations can move independently or worse, exist in name alone and be restricted by Technology's policies. In regard to Data and Product, we have observed two main factions, Product members that understand that scalability is directly tied to Data and those that show no care to the tight connection of their own technical debt. A two dimensional problem becoming three.
We commonly talk about technology debt ("Tech Debt"), but rarely do we discuss that most of this debt is tied specifically to the underlying architecture, which is typically data related topics, e.g., data modeling, service orienting reference data. Why is this? Strong advocates on the data side ask, "How do you expect to scale your product when you build top/down (front end to back end to data)?". It is universally agreed in the field of Computer Science that building bottom/up (data to back end to front end) helps scale applications and minimizes technical debt. The reward of strong architecture results in more opportunities to empower business value. This observation excludes the common sense approach when prototyping, as your viable product may not need scale.
The logical questions that everyone then looks to solve is, "What if you don't start with a focus on managing Data Debt? What can be done to shift organizational culture?".
From solving this issue at the largest Fortune 500 companies to the smallest startups, we have found several recurring themes for companies that successfully manage their data. We will be looking at common themes and principals for positive cultural contributions and why many fail in their journey.
The primary way data organizations communicate a consistent present and future data roadmap has always been the Entity Relationship Diagram (ERD). A key tool for not only explaining how data is stored or used, but can be a central resource for an organizations way to standardize language and explain how processes and systems work.
In the image above we have a system that is comprised of various Investments in Funds. In order to have an Investment, an Account needs to be created by an Investor. An Investor can have many Accounts. Those Accounts may or may not be advised by an Advisor.
This ERD uses simple language, reflects how both technical and non-technical individuals in the company would talk to processes. This is easy to print and distribute. It requires no technical skill to understand.
A concept may be introduced that will require a change in how we define our system/process. In the example above, the Team is introduced. Teams allow multiple Advisors to manage an Account when relevant. Having this transparency allows all individuals within an organization to understand what is possible. Everyone can easily understand a Team and what it is used for. It doesn't matter if you are writing SQL as a developer or managing the relationship with the Advisors directly. To go a step further the physical dimensions/tables within the database can be named to reflect this shared language.
Having the ERD isn't enough. Leaders in the company need to not only encourage the discussion of data regularly, but use large touch points, like town halls, to market the importance. Print and post the one pager (ERD) everywhere.
Data organizations should never own the data. They should build frameworks for Technology/Product teams to manage their applications/services. Business domains need to own the underlying data with the support of their associated Product partners. A lot of times, Product organizations only focus on the delivery of data through front end user interfaces. This is problematic for two reasons:
The business domain knowledge will always be stronger in the aligned Product team. Ultimately, this mindset will allow Product teams to build tools that directly charge the Business to have full control over their data.
The assumption in most companies is Business teams teach Product domain knowledge. This relationship, although not one sided, due to Products creation of value through the tooling it creates, could go a step further to impart technical knowledge on their partners. Treat your business with respect and assume they can learn technology and data concepts. You will likely come across some users that don't care, but you miss all goals if you never take a shot. Even if they don't care, we guarantee you'll find someone near them that does.
Topics to teach:
Culture takes time to build, but always produces dividends.