Exploring the future of financial data architecture: AI-native financial data foundation, large-scale financial data processing, and enterprise data quality & governance.
Tag: Data Architecture
Before I start talking about how effective this architecture can be at reducing infrastructure costs, I should first make the old point that there is really no free lunch. Compared with commercial cloud data platforms and warehouses such as Databricks, BigQuery, and Snowflake, an open lakehouse setup requires significantly more engineering effort to build, operate, … Continue reading S3 + Parquet + Iceberg + Trino: A Poor Man’s Market Data Platform
For many years, a large portion of systematic strategies relied on relatively low-frequency signals. These approaches worked well when they were under-explored, but over time they have been widely researched, increasingly arbitraged, and structurally compressed in edge. As a result, a growing share of remaining opportunity has shifted toward market microstructure — order flow dynamics, … Continue reading How QuantFlow Handles Large-Scale Market Data
What makes me buy into DolphinDB: Friendly DolphinDB – Cross-Exchange Arbitraging Case Speedy DolphinDB – Why is DolphinDB so fast? Robust DolphinDB – Reliable, Scalable, Resilient, Consistent, and Monitorable Cost Effective DolphinDB – Worth the Money DolphinDB – An Integrated Financial Data Platform, Not Just a Time-Series Database When I chat about DolphinDB with someone, it’s common … Continue reading DolphinDB – An Integrated Financial Data Platform, Not Just a Time-Series Database
What makes me buy into DolphinDB: Friendly DolphinDB – Cross-Exchange Arbitraging Case Speedy DolphinDB – Why is DolphinDB so fast? Robust DolphinDB – Reliable, Scalable, Resilient, Consistent, and Monitorable Cost Effective DolphinDB – Worth the Money DolphinDB – An Integrated Financial Data Platform, Not Just a Time-Series Database This is the fourth post in my series where … Continue reading Cost Effective DolphinDB – Worth the Money
What makes me buy into DolphinDB: Friendly DolphinDB – Cross-Exchange Arbitraging Case Speedy DolphinDB – Why is DolphinDB so fast? Robust DolphinDB – Reliable, Scalable, Resilient, Consistent, and Monitorable Cost Effective DolphinDB – Worth the Money DolphinDB – An Integrated Financial Data Platform, Not Just a Time-Series Database As a high-performance database built for business-critical financial applications … Continue reading Robust DolphinDB – How does DolphinDB Achieve Scalability, Reliability, Resilience, Consistency, and Monitorability
Buy-Side Financial Data Engineering (1) - Overview Buy-Side Financial Data Engineering (2) - Financial Instruments Buy-Side Financial Data Engineering (3) - Market Data Management As a data guy, two thoughts immediately come to my mind when I hear the term "Finance Market Data", 1) They are bloody expensive; 2) What a chore to handle all … Continue reading Buy-Side Financial Data Engineering (3) – Market Data Management
Buy-Side Financial Data Engineering (1) - Overview Buy-Side Financial Data Engineering (2) - Financial Instruments Buy-Side Financial Data Engineering (3) – Market Data Management The second article of my "Buy-Side Financial Data Models" focuses on the "Financial Instruments" data domain. Financial instruments data is complex and difficult to manage. In the meantime, it is crucial to … Continue reading Buy-Side Financial Data Models (2) – Financial Instruments
Buy-Side Financial Data Engineering (1) - Overview Buy-Side Financial Data Engineering (2) - Financial Instruments Buy-Side Financial Data Engineering (3) – Market Data Management This is the first blog post of the "Buy-Side Financial Data Models" series I am planning to write. To kick off this blog series, this post provides a high-level overview of the … Continue reading Buy-Side Financial Data Models (1) – Overview
"What is data management?". I guess many people will (at least I think I will) answer "em... data management is managing data, right?" at the same time swearing in their heads that "what a stupid question!". However, if I was asked this question in a job interview, I guess I'd better to provide a bit … Continue reading What is Data Management, actually? – DAMA-DMBOK Framework
Microsoft claims Azure Storage providing both high availability and strong consistency. It sounds good but obviously violates the CAP theorem as the 'P' (network partitioning) is not avoidable in the real world. In theory, you can only achieve either high availability or strong consistency in a distributed storage system. I have done a bit of … Continue reading How Azure Storage Cheats Over the CAP Theorem
Data Ninjago (Financial Data Architecture) 
You must be logged in to post a comment.