·
7 min read

Harness the future with the ultimate hybrid platform for data and AI

Today I’m excited to give the Day 1 keynote at PASS Summit v.20, a gathering of our longtime community of SQL Server users and data professionals.  PASS Summit is an amazing chance to see the faces of old and new friends.  It’s a place to meet with customers and fans to continually learn about their evolving needs and to help us grow as a SQL community and develop the best data platform products in the market.

Hybrid connects all your data

Now more than ever, we are architecting for hybrid, because we are hearing from customers that they will be running data workloads on-premises and in the cloud – rarely just one or the other. We believe that the value Microsoft can add is to provide a great and consistent experience wherever they deploy.  One example of this commitment is Azure SQL Database Managed Instance, which was recently made generally available.  Managed Instance enables organizations to migrate their SQL Server workloads to Azure with zero code changes and offers an easy path to the cloud at an incredible value – and with security, intelligent performance and management tools that are unique to our cloud database services.  The end of extended support for SQL Server 2008 and 2008 R2 next year is a great opportunity for customers to rehost to Azure SQL Database Managed Instance, a fully-managed solution that eliminates the need for future upgrades. It’s easy to get there using comprehensive, yet easy-to use migration tools like Azure Database Migration Service.

Microsoft is excited to announce the general availability of Azure SQL Database Managed Instance Business Critical tier on December 1. Designed for mission-critical business apps with high I/O requirements, the business critical tier supports high availability with the highest level of storage and compute redundancy.  This new tier provides support for in-memory processing, a range of sizes up to 80 cores on Gen5, and zone-redundant HA using several isolated replicas to provide the highest resilience to failure.  We offer all this performance at an incredibly compelling price point—up to 85% less expensive than AWS.  Programs such as the Azure Hybrid Benefit, which allows customers to re-use their on-premises SQL Server licenses for discounts in Azure, and upcoming Reserved capacity pricing for Managed Instance Business Critical which allows you to prepay for a 1 or 3-year term commitment, further help you manage costs in the cloud.

In addition, we’re announcing a limited preview of Machine Learning services in Azure SQL Database. You can now use the Azure SQL Database support for Microsoft Machine Learning Services with R language to complete data processing, model training, and scoring all inside your SQL Database. This means you no longer need to move data out of the database to train and operationalize machine learning models. The R code can be deployed in production by embedding it in T-SQL stored procedures.

Azure is also a great destination for open source database migration. We recently announced an expansion to our relational OSS database managed service offerings with a preview of Azure Database for MariaDB. With MariaDB joining MySQL and PostgreSQL, Azure now offers the community versions of all the most popular OSS relational databases as managed services in the cloud, with advantages like built-in high availability, dynamic scaling and world-class security features.  When looking to migrate NoSQL databases like MongoDB and Cassandra to the cloud, Cosmos DB provides an excellent destination.  Customers gain access to managed NoSQL at a lower total cost of ownership (TCO) vs. on-premises and cloud competitors not to mention the industry leading SLA, global distribution, and features that take all the work out of DevOps. You can get started today by using the Azure Cosmos DB API of your choice to migrate NoSQL data and apps from MongoDB, Cassandra, Hbase, and more, using a free trial of Azure Cosmos DB.

With SQL Server 2019, organizations can now seamlessly manage their structured and unstructured data in a single, integrated solution. It comes with big data capabilities built-in, including support for Apache SparkTM and Hadoop Distributed File System (HDFS)—everything you need to build a data lake with your SQL Server skills. Today we announce SQL Server 2019 community technology preview (CTP) 2.1 which has a number of new features for the database engine and big data clusters:

  • Ability to deploy R and Python apps inside a SQL Server big data cluster
  • Scalar UDF inlining feature in Intelligent query processing, optimizing a common performance problem scenario for User Defined Functions
  • Derived table or view aliases in graph match queries
  • Improved diagnostic data for long-running queries, helping to pinpoint when a query is blocked by stats background processing
  • Ability to put buffer pool in persistent memory, dramatically speeding up I/O operations

And we have more planned for upcoming previews of SQL Server 2019, including Accelerated Data Recovery to speed up recovery processing, transaction rollback, readable secondaries, and adding availability groups for system databases which enables users to replicate linked server definitions, logins, and SQL Agent jobs to the secondary replicas.

Hybrid enables comprehensive AI and analytics

Having the most consistent data platform across on-premises and cloud enables us to offer AI and analytics over all your data. Azure SQL Data Warehouse is a cloud data warehouse that combines lightning fast query performance with advanced security features to turn all your data into actionable insights. Azure SQL Data Warehouse has been recognized as the fastest cloud data warehouse by third party benchmarks. Building upon its industry leading performance, today we announced significant security and usability updates. Customers can now take advantage of a new workload importance feature to influence query execution by priority, making sure high business value work gets first access to system resources. In addition, SQL Data Warehouse now offers native row level security, enabling customers to implement the most stringent security policies for fine-grained access control. Other new capabilities include support for SQL Server Data Tool, enhanced performance monitoring, advanced tuning and accelerated database recovery to significantly improve service usability. Learn more about this and other new features in today’s Azure SQL Data Warehouse blog. Experience the performance of Azure SQL Data Warehouse by creating your first data warehouse.

Azure SQL Data Warehouse also offers efficient and scalable structured streaming write support through native Azure Databricks connector. Azure Databricks is an Apache® Spark™-based analytics platform that enables you to accelerate and simplify the process of building big data and AI solutions to drive the business forward, all backed by industry leading SLAs. We recently announced the preview of Azure Databricks Delta, a powerful transactional storage layer built on Apache Spark to provide better consistency of data and faster read access. Organizations also benefit from Azure Databricks’ native integration with other services like Azure Blob Storage, Azure Data Factory, and Azure Cosmos DB. This enables new analytics solutions that support modern data warehousing, advanced analytics, and real-time analytics scenarios.

Announced at Ignite, the Azure Data Explorer preview is a fast, highly scalable data exploration service for log and telemetry data. It helps you handle many data streams, so you can collect, store, and analyze data. Azure Data Explorer is ideal for analyzing large volumes of diverse data from any data source, such as websites, applications, IoT devices, and more. Azure Data Explorer makes it simple to ingest this data and enables you to perform complex ad-hoc queries on the data in seconds.  Today at PASS, we’re excited to host one of our customers, Taboola, to demonstrate how their organization is using Azure Data Explorer to analyze web data in near real time, crunching large amounts of web data in order to provide the best content recommendations to webpage readers – all in real time.

Customers who want to stream data or analyze in real-time to get valuable insights faster need a massively scalable, distributed, event-driven messaging platform with multiple producers and consumers. Apache Kafka and Azure Event Hubs provide such distributed platforms.  Azure Event Hubs for Apache Kafka, now generally available, provides a Kafka endpoint that can be used by your existing Kafka-based applications as an alternative to running your own Kafka cluster. With Event Hubs for Kafka, you get the best of both worlds—the ecosystem and tools of Kafka, along with Azure’s security and global scale—in a fully managed solution.

Experience your data

Microsoft’s Business Intelligence (BI) tools are evolving as well. Power BI already includes robust self-service data preparation capabilities in Power BI Desktop through the familiar Power Query based experiences that are used by millions of users worldwide. With the new public preview of dataflows in Power BI, we’re taking self-service data preparation to the next level, enabling business analysts to create data preparation logic that can be reused across multiple Power BI reports and dashboards and linked together to create sophisticated data transformation pipelines. Dataflows can be configured to store the data in the customer’s Azure Data Lake Storage Gen2 instance, and dataflows support the Microsoft Common Data Model, giving organizations the ability to leverage a standardized and extensible collection of data schemas (entities, attributes, and relationships).

For our long-time BI customers with investments in SQL Server Reporting Services, with the public preview today, you can include pixel-perfect paginated reports alongside to Power BI’s existing interactive reports. This provides a unified, secure, enterprise-wide reporting platform accessible to any user across devices. You can read more about these innovations in a blog from Arun Ulagaratchagan, General Manager, Power BI Engineering.

Getting started

In conclusion, I’m excited to share with you that hybrid is here:  Microsoft’s consistent data platform across on-premises and cloud connects all your data and makes intelligence over all your data possible. We are proud to provide customers with the widest range of options to run SQL on Azure at the best price.  And our best-of-breed data analytics options bring AI to all your data.

  •  If you’d like to watch my talk at PASS, you can sign in on the PASS website.  Registration is free and sessions the keynote content starts at 8:15 AM Pacific.
  •  If you’re ready to get started, here are a few great places to get going:

 

Rohan Kumar posing for the camera
Rohan Kumar
Corporate Vice President, Azure Data

Related Posts