How banking analytics can capture the moment with live data
Andrea Braida of WANdisco: Hi Pranav! What is the financial services sector doing with hybrid cloud and banking analytics? What is the mindset and how do they think in strategic terms?
Pranav Rastogi of Microsoft: I think the move to the cloud is last year’s story. Today’s CDO’s are looking to exploit cloud capabilities, which means real-time data while enjoying the same protections and governance as traditional data. The opportunity is right there for Microsoft Azure to demonstrate its awesome cloud capabilities, and WANdisco to provide the enabling live data environment.
It’s no longer good enough to deliver historic data analysis: all the action is out there at the real-time edge. As data ages, it loses value; data in real time represents money, because you can make decisions – maybe anti-fraud at the transaction level, maybe strategic at the corporate level – at the moment that the data is generated.
Live data everywhere really creates one thing: value. This is a huge step forward for financial institutions, but presents a big problem: How do they keep a complete, integrated view of their data, divided between the cloud and on-prem?
Andrea: Exactly. Can you help me understand which technologies are really driving data agility?
Pranav: The biggest change came with cloud, which suddenly made it easy for customers to experiment. Azure provides a broad set of IaaS, PaaS, and SaaS offerings, which allows customers to focus on the code without having to worry about the infrastructure.
The combination of the speed of setting up new services plus the need to run banking analytics as close to data as possible resulted in a rapid, and huge, move to the cloud. Financial institutions have deployed solutions built on machine learning and AI that offer fraud detection and automated remediation right where the data is generated even at the edge. This is very different from transferring data back to the center for analysis, reporting, and sending back the result to the origin.
For machine learning and AI-based solutions to become truly effective, however, they need good training data – otherwise they will fail to detect fraud or produce false positives, and banks and consumers will lose trust in the system fairly quickly. So financial institutions must ensure that the training data is valid and accurate. With the proliferation of devices and transactions, banks are capturing and managing exabytes of data.
How do banks ensure that they have a single version of the truth across all of the sources and destinations that exist including the challenges of hybrid-clouds? For the data to be usable and provide a rich source for predictive analytics and AI training, it must be synchronized in some manner across all these sources. There is an immediate need for a solution that ensures data is synchronized across systems regardless of on-prem, cloud, and devices, even during migrations, and also enables full business continuity and disaster recovery (BCDR).
Andrea: Would you say that these challenges stop financial institutions from exploiting the cloud?
Pranav: I’m seeing customers struggle. Financial institutions have figured out the capabilities that cloud offers. The first cloud journey is lifting and shifting current applications to take advantage of cloud capacity, scalability and more. The second journey concerns digitally transforming their business through banking data analytics, which is the more critical part of the workload.
In moving banking data analytics workloads, the key challenge is how quickly they can move – and that’s where they struggle. How can they replicate these huge, huge quantities of data in real-time?
As we speak, I think the process is fairly involved, with a typical analytical application proof-of-concept taking some three to six months. One of the hardest parts is figuring out which data sets to move and ensuring that the integrity of those data sets is preserved as the data is being replicated from their environment to the public cloud.
Andrea: Well a six-month proof of concept sure takes the shine off the speed and ability of the cloud. How are people cutting that delay?
Pranav: The ‘live data’ concept is the first real breakthrough idea that accelerates the move to cloud. Having data fully available, having the same data that you have on-prem and always in sync, creates the ability to exploit new ideas on the cloud. Live Data removes a major block from executing a proof of concept: the data is already there – valid, accurate, and ready to run. HDInsight Application Platform allows customers to deploy a secured WANdisco Fusion instance using the Azure Marketplace to reduce the proof of concept time.
Andrea: As well as solving the technical challenges around data replication, how else can the ‘live data’ concept assist financial institutions? I’m thinking here of regulatory compliance, including availability and business continuity.
Pranav: I would argue that data is secondary to the application environment, governance and policies. Does the cloud meet those standards? Azure is in a good space for financial institutions because it meets data residency, sovereignty, compliance, and resiliency requirements are honored within geographical boundaries especially for analytics.
While it is important that you have Live Data available everywhere, it’s also important to ensure that you have live policies available, across all the environments that you have deployed. And those policies must meet business continuity and disaster recovery (BC/DR) requirements.
WANdisco Live Data supports replication of Live Data to several Microsoft analytical services including HDInsight, Azure Data Lake Store, Azure Storage, and more. WANdisco supports replication for the most popular open source ecosystems such as Apache Hadoop, Spark and more.
My conclusion is that live data can apply to data, application, metadata, policies, and more, to allow financial institutions to build enterprise solutions in the cloud faster. As I said before in the example of fraudulent transactions, you can lose trust in a system very rapidly, and governance is therefore a critical part of the mix.
Andrea: Thanks for your time today!
Pranav: Sure thing. You can learn more about continuously available data by reading my recent blog on this topic. Thanks for the interview!
Check out the Azure blog
