Today, we have some amazing new tools for building a modern data platform. But most of us don’t have the luxury of pausing everything to implement them. In a fast-paced world, we have to keep systems running, deliver new features, and adapt to constant demands—all while transforming to a platform that’s more productive and efficient. This balancing act rarely follows a perfect plan. Based on my experience, here’s a practical approach to managing the core challenges effectively.
1. Start without waiting for perfection
Don’t wait for the perfect roadmap. Begin with a broad vision and adjust along the way. Early wins help keep momentum and build trust.
Action: Set clear, short-term goals based on outcomes rather than over-detailed plans. Start, then refine as you go.
2. Set strong foundations and protect them
Laying the groundwork is essential. Ensure dedicated capacity, skills, and time to establish data structures and governance. Protect this work from reprioritisation, even when other demands come up.
Action: Assign a team to foundational tasks and maintain their focus. Don’t reprioritise this work once it’s underway.
3. Manage complexity by ring-fencing legacy
Legacy systems add complexity and technical debt. Simplify by ring-fencing only what’s essential, and keep the new platform’s architecture lean.
Action: Identify essential legacy functions to ring-fence, while keeping the new platform uncluttered.
4. Be use case-driven with data modelling
In the past, we defaulted to building complex data models. Now, focus on loading raw data and transforming only for specific use cases. This keeps efforts purposeful.
Action: Transform data only when there’s a direct use case. Avoid overcomplicating data models without clear value.
5. Manage your costs: avoid a platform black hole
With multiple platforms, costs can spiral out of control. Evaluate platform usage to prevent runaway expenses and avoid duplicating functionality.
Action: Regularly assess which platforms deliver real value. Consolidate where possible to keep costs manageable.
6. Know your tools and use them intentionally
New platforms come with a range of options, but not every feature is necessary. The days of 50-column reports are over; we need purpose-built tools that prioritise insights.
Action: Choose tools wisely, optimising for relevance and simplicity. Move away from outdated practices like excessive column reports.
7. Balance new demands without adding chaos
Transformation brings constant demands, but responding to each one can lead to “spaghetti architecture.” To avoid this, we need clear guiding principles and standards. Prioritise data security, data accuracy, and standards alongside data quality to ensure sensitive information is managed securely and consistently.
Action: Set a roadmap that balances quick wins with long-term goals, maintaining high standards for data security, accuracy, and quality. Regular alignment sessions help manage stakeholder expectations and keep everyone focused on mission-critical objectives.
8. Build a community for upskilling and collaboration
A strong community drives adoption, upskilling, and collaboration, especially between technology and data teams. Effective collaboration ensures that technical and data insights align, creating solutions that serve the business better. It’s also essential to keep an eye on new features that could boost productivity, balancing when to introduce these enhancements to maximise impact without disrupting workflow. Encourage knowledge-sharing across teams to support learning and foster a data-first culture.
Action: Organise regular upskilling sessions and appoint “data champions” in each team to support ongoing learning. Promote collaboration between technology and data teams to bridge gaps, build a cohesive data-driven community, and identify the right time to bring in new productivity-enhancing features.
9. Focus on adoption and culture change
Transformation only succeeds if people actually use the new platform. Old systems stick around out of habit, so self-service tools and training bridge the gap.
Action: Roll out features gradually and prioritise data literacy. With “data champions” in place, adoption is smoother and a data-driven culture can grow.
10. Design for AI readiness
AI and machine learning play an increasingly central role in data strategy. As you modernise your platform, think ahead to ensure it’s AI-ready. Centralise data sources, streamline workflows, and create infrastructure that can scale with AI demands. An AI-ready platform will help future-proof the investment, enhancing both productivity and business insights.
Action: Plan for centralised, scalable ML capabilities and infrastructure that supports AI integration. This will ensure a platform that can easily accommodate advanced analytics and emerging AI technologies as your data needs evolve.
In summary
Transforming legacy systems to a modern data platform is more than a technology upgrade. It’s about building a focused, adaptable platform for real use cases while keeping costs under control. By setting strong foundations, using tools wisely, managing costs, encouraging upskilling, and phasing out old practices, we can drive lasting transformation and real value.
Our recent tech blog posts
Transformation is for everyone. We love sharing our thoughts, approaches, learning and research all gained from the work we do.
Find out about the challenges of enabling data discovery and the need to recognise the efforts of data publishers to enable data findability.
Read more
From SQL Server, to Azure, to Fabric: My Microsoft Data and Analytics journey
There is always something new in the Microsoft space, and Fabric is the latest game-changing update to impact the world of analytics.
Read more
Predicting and monitoring air quality
How we delivered an end-to-end forecasting solution for predicting the values of the air pollutant, PM2.5, in Newcastle.
Read more
