A Unified Real-Time Dashboard for Data Excellence
Client
Netvista Systems LLC
Project Duration
2 months sprints
Team
Daniel Rosenberg ( Mentor )
Netvista Team
My Role
Research, Persona, Dashboard Overview Design, Prototyping, Testing.
Tools used
Figma / Figjam
Axure rp
Miro
Objectives & Challenges
Objectives
​
Deliver actionable, real-time data visualizations.​
​
Enable rapid decision-making with a unified global and local view.​
​
Empower data engineers to prioritize critical tasks.
​
Challenges
​
Complex Monitoring:Overwhelming data streams create difficulty isolating issues.
​
Delayed Alerts: Traditional tools lead to slow responses.
​
Information Overload: Unfiltered data obscures key metrics.Limited Customization: One-size-fits-all views don't meet user needs.
User Persona
-
Deliver value to the company​​​
-
Improve operational efficiency
-
Ensure data consistency, security, accuracy, and timeliness
-
Minimize time & costs spent on data pipelines
-
Automate repetitive tasks
-
Uplevel the business with new technologies
Name: John Davis
Role: Data Engineer
Experience
-
BS/MS in CompSci, InfoSystems.
-
Statistics5+ years as Data Engineer
-
Experienced with SQL and NoSQL databases, data pipelines, data warehouses, machine learning, cloud services, and scripting.
-
Deliver value to the company​​​
-
Improve operational efficiency
-
Ensure data consistency, security, accuracy, and timeliness
-
Minimize time & costs spent on data pipelines
-
Automate repetitive tasks
-
Uplevel the business with new technologies
Responsibilities
Needs and Movitvations
User Pain Points
Monitoring
complexity
Data overload
Users find monitoring multiple data streams challenging due to the complexity and volume of data.
Delayed response
Current tools do not provide real-time alerts, resulting in delayed problem responses.
Too much unfiltered data leads to difficulty in identifying critical issues.
Lack of Customization
Users require customizable views to focus on metrics relevant to their needs.
Competitive Market Research
To understand industry benchmarks and gaps, I conducted a competitor analysis of leading data integration platforms:
Competitor | Key Limitations |
|---|---|
SnapLogic | Lacks intelligent filtering, leading to data overload. |
MuleSoft | Switching between different monitoring tools is time-consuming. |
Hevo | Customization options are restricted, preventing user adaptability. |
FiveTran | Limited real-time visualization, making proactive decisions difficult. |
Informatica | Lacks a unified dashboard integrating global and local health metrics. |
Key Findings:
​
Lack of unified system monitoring: Users often switch between tools to analyze performance.
Delayed real-time insights: Many competitors lack immediate anomaly detection.
Restricted customization: Users cannot adjust dashboards based on individual workflow needs.
Opportunity: Design a solution that integrates real-time analytics, anomaly detection, and user customization into a single, intuitive dashboard
My Approch For Global Dasboard
Determine the aggregate health of all Apps in the production environment​.
​
Enhance system reliability, minimize downtime, and prioritize critical tasks​.​
​
Comprehensive visualizations highlight Apps not meeting checkpoints, provide insight on Apps with the most lag, and enable preventive actions or debugging.
Conceptual Model
To break down the information heavy content into simpler navigational attributes.
Transactional Attributes:
Text | Application | Checkpoints | Alerts |
|---|---|---|---|
Cluster name | Developer's name | Events | App |
App | Check Points | Time | Node Cluster |
Status | Destination | Status | Date and Time |
App name | Events | ||
Source |
Analytical Attributes List:
MEASURE DIMENSION MATRIX: For easier segregation of quantitative data into graph & determining x-axis v/s y-axis
Node Cluster | Measure | Unit/Value/Dimension |
|---|---|---|
Statistical Analysis
| Quantity: Volume of space | Server memory consumption; CPU consumption; Total percentage of capacity taken |
Stability | Stability: Status change | Frequency of change in status |
Velocity: Speed of processing | Average speed of events being processed; average lag time | |
Quantity: Number of events being processed | Event processed |
Applications | Measures | Unit/Value/Dimension |
|---|---|---|
Statistical analysis | Quality: App performance | Read; Write; Read & Write Dev.; Backpressure; App rate |
Stability | Quantity: Volume of storage | Server memory consumption; CPU consumption; Namespace Consumption; Total percentage of capacity taken |
Stability: Status change | Frequency of change in status; Number of check points reached
| |
Velocity: Speed of processing | Average lag time
| |
Quantity: Number of events being processed | Event Processed |
Check points | Measure | Unit/Volume/Dimension |
|---|---|---|
Statistical Analysis | Velocity: Speed of processing | Average time from beginning to end of each checkpoint |
Quantity: Volume of space | How much volume does each checkpoint use ( CPU, Memory, Namespace consumption) |
Events | Measure | Unit/Volume/Dimensions |
|---|---|---|
Statistical Analysis | Quality: Performance | Read & Write Devation; Backpressure; Forecast |
Data Pattern | Write Quantity: Volume of space | How much volume events take |
Read | Velocity: Speed of processing | Average speed events being processed |
Write | Quantity: Number of events being processed | Events Processed |
Event |
Alert | Measure | Unit/Volume/Dimension |
|---|---|---|
Stability | Status change | Volatility of status |
Strategy Approch & Design Execution
Ben Shneiderman's visualization theory:
​
-
More effective detection of faulty data, missing data, unusual distributions, and anomalies​​.​
-
Deeper and more thorough data analyses that produce profounder insights​​.
-
Richer understandings that enable researchers to ask bolder questions
Overview of key metrics and KPIs in a digestible, easily comprehensible manner using charts, graphs, and other visual aids relevant to the needs of data engineers.
Node cluster Performace Physical actions or presses of labeled interface objects (e.g. buttons, sliders, etc.) instead of complex syntax. Flexibility in sorting drop down menu helps to identifying environments and nodes for multiple Apps
Deeper and more thorough data analyses that produce profounder insights​​.
Impact & Measurable Outcomes
58%
Increase in User efficieny
66%
59.9%
Reduction in Downtime
Faster issues resolution
The intuitive dashboard not only streamlined system monitoring but also empowered data engineers to prevent performance bottlenecks before they escalate.
Key Learnings & Future Directions
Lessons from the Process
​ Real-time Insights are Critical – Instant alerts drive proactive decisions.
Balancing Complexity with Simplicity – A clear visual hierarchy enhances usability.
Customization is Non-Negotiable – Users demand flexible dashboard layouts.
​​
Future Enhancements
AI-powered Anomaly Detection – Predict failures before they happen.
Advanced Customization – Allow engineers to create personalized workflows.
Collaborative Troubleshooting – Improve team visibility for shared problem-solving.
