21 August 2016

Why Cortana Analytics Suite




Cortana Analytics Suite (CAS), what can it do for you


Microsoft introduced the Cortana Analytics Suite (CAS) in July 2015, at the Worldwide Partner Conference in Orlando. Want to learn more then read on.

Cortana Analytics Suite


When Microsoft first announced CAS, it touted the suite as an integrated set of cloud-based services that vaguely promised to be “a huge differentiator for any business.” The suite would be available through a simple monthly subscription and be customizable to fit the needs of different organizations. The company planned to make CAS available that coming fall.

Two months later, Microsoft hosted the first-ever Cortana Analytics Workshop, a gathering of techies that would provide participants with a chance to learn about Microsoft’s advanced analytics vision. The workshop appeared to represent the suite’s official launch.

Microsoft Envision | Impactful analytics using the Cortana Intelligence Suite with EY


At some point during the build-up, Microsoft also set up a slick new website dedicated to the CAS vision ( https://www.microsoft.com/en-us/server-cloud/cortana-analytics-suite/). The website featured rolling graphics with stylized icons, and large bold headlines that emphasized the suite’s imminent importance. Cortana Analytics, it would seem, had officially arrived.

As we can see from the above architecture diagram, following are the key pillars of Cortana Intelligence Suite:



Information Management: Consists of services which enable us to capture the incoming data from various sources including the streaming data from sensors, devices, and other IoT systems.  Manage various data sources which are part of the data analytics ecosystem within the enterprise; and orchestrate and build end-to-end flows to perform various activities and data processing and data preparation operations.
Big Data Stores: Consists of services which enable us to store and manage large scale data. In other words, enables us to store and manage big data. These services offer high degree of elasticity, high processing power, and high throughput with great performance.
Machine Learning and Analytics: Consists of services which enable us to perform advanced analytics, build predictive models, and apply machine learning algorithms on large scale data.  Allows us to perform data analysis on large scale data of different variety using programming languages like R and Python.
Dashboards and Visualizations: Consists of services which enable us to build reports and dashboards to view the insights. It primarily consists of Power BI which allows us to build highly interactive visually appealing reports and dashboards. Apart from this, other tools like SQL Server Reporting Services (SSRS), Excel, etc. can also be used to connect to data from some of these services in Cortana Intelligence Suite.
Intelligence: Consists of advanced intelligence services which enable us to build smart interactive services using advanced text, speech, and other recognition systems.

  • “Take action ahead of your competitors by going beyond looking in the rear-view mirror to predicting what’s next.”
  • “Get closer to your customers. Infer their needs through their interaction with natural user interfaces.”
  • “Get things done with Cortana in more helpful, proactive, and natural ways.”

Modern Data Warehousing with the Microsoft Analytics Platform System

Cortana Intelligence Suite Highlights

Here are the highlights of Cortana Intelligence Suite:

  • A fully managed Big Data and Advanced Analytics Suite enabling businesses transform data into intelligent actions.
  • An excellent offering perfectly suited for handling modern day data sources, data formats, and data volumes to gain valuable insights.
  • Offers various preconfigured solutions like Forecasting, Churn, Recommendations, etc.
  • Apart from the big data and analytical services, Cortana Intelligence Suite also includes some of the advanced intelligence services - Cortana, Bot Framework, and Cognitive Services.
  • Contains services to capture the data from a variety of data sources, process and integrate the data, perform advanced analytics, visualize and collaborate, and gain intelligence out of it.
  • Offers all the benefits of Cloud Computing like scale, elasticity, and pay-as-you-go model, etc.

Microsoft Envision | Running a data driven company

Use Cases for the Cortana Intelligence Suite

Cortana Intelligence Suite can address the data challenges in various industries and enable them to transform their data into intelligent actions and helps to be more proactive in the day-to-day operational aspects of the business. Of the various industries where Cortana Intelligence Suite can be used, here are a few of them.

Financial Services: Monitor the transactions as they happen in near real-time and based on the analysis on the historical data and historical data anomalies/trends, Cortana Intelligence Suite can be used to apply complex machine learning algorithms and predictive models to predict a potential fraudulent transactions and help business prevent such transactions in future thereby protecting customer's valuable money. The Financial Services sector is pretty vast and we can use Cortana Intelligence Suite in various scenarios including credit/debit card fraud, electronic transfer fraud, phishing attempts to steal confidential customer data, etc.
Retail: Cortana Intelligence Suite can be used across the Retail Industry in various scenarios including optimizing availability by forecasting demand, enabling businesses to ensure the right products in the right location at the right time. There are numerous use cases in the retail industry and Cortana Intelligence Suite can be used in conjunction with IoT systems. For instance, with the help of sensors (Beacon Technology) we can detect when a customer enters a retail store and based the data that we have in the database about that customer, we can offer them targeted discounts based on customer's demographics, past purchase history, what the customer has been browsing online (this is where bringing in the data from outside the enterprise comes into picture as discussed in this tip on Introduction to Big Data), and other relevant information which can help understand the customer's preferences.
Healthcare: There are various scenarios in Healthcare where the Cortana Intelligence Suite can be used. Historical data on the utilization of various resources (Rooms, Beds, Other Equipment, etc.) and manpower (Doctors, Nurses, general staff, etc.) can be analyzed to predict the future demand thereby enabling the hospitals to mobilize and optimize the resources and manpower accordingly. Historical patient data can be analyzed in conjunction with weather data to identify the patterns and potential illness that might be caused during particular seasons and help the authorities take preventive measures.
Manufacturing: By constantly monitoring the equipment and collecting the data over time, probability of issues occurring can be predicted and accordingly a maintenance schedule can be defined to prevent the potential issues which if occur can hamper the production and day-to-day operations leading to unhappy customers, loss of business, and increased operational costs. Cortana Intelligence Suite fits very well in this scenario and enables end to end data collection, monitoring, alerting, and to take proactive actions/decisions.
Public Sector: There are various areas in the public sector where Cortana Intelligence Suite can be used to improve the overall operational efficiency including Public Transport, Power Grids, Water Supplies, and a lot more. By monitoring the usage of resources in various areas, we can identify the patterns in the usage, predict and forecast the demand, and accordingly ensure the supply so that there is neither shortage nor a waste of resources thereby improving the overall operational efficiency and happy customers.

Microsoft Envision | ZAP presents: It’s all about the data --big, small, or diverse



Above are just a glimpse of scenarios in each of those sectors and there are many more such scenarios in each of the sectors. Apart from these, there are various other countless sectors where the Cortana Intelligence Suite can be used like Education, Insurance, Marketing, Hospitality, Aviation, Research, and so on.

The Azure side of Cortana Analytics Suite



When it comes to the individual Azure services, we can often find more concrete information than we can with Cortana Analytics. That’s not to say we won’t run into the same type of marketing clutter, but we can usually find details that are a bit more specific (even if it means going outside of Microsoft). What we don’t find are many references to Cortana Analytics, although that doesn’t prevent us from building the types of solutions that the CAS marketing material likes to show off.



The first of the CAS-related services have to do with storing and processing large sets of data:

Azure Data Warehouse : A database service that can distribute workloads across multiple compute nodes in order to process large volumes of relational and non-relational data. The service uses Microsoft’s massive parallel processing (MPP) architecture, along with advanced query optimizers, making it possible to scale out and parallelize complex SQL queries.

Azure Data Lake Store: A scalable storage repository for data of any size, type, or ingestion speed, regardless of where it originates. The repository uses a Hadoop file system to support compatibility with the Hadoop Distributed File System (HDFS) and offers unlimited storage without restricting file sizes or data volumes.

Azure Data Lake Store is actually part of a larger unit that Microsoft refers to as Azure Data Lake. Not only does it include Data Lake Store, but also Data Lake Analytics and HDInsight, both of which share the CAS label. You can find additional information about the Data Lake services in the Simple-Talk article Azure Data Lake.

The next category of services that fall under the CAS umbrella focus on data management:

Azure Data Factory : A data integration service that uses data flow pipelines to manage and automate the movement and transformation of data. Data Factory orchestrates other services, making it possible to ingest data from on-premises and cloud-based sources, and then transform, analyze, and publish the data. Users can monitor the pipelines from a single unified view.

Azure Data Catalog : A system for registering enterprise data sources, understanding the data in those source, and consuming the data. The data remains in its location, but the metadata is copied to the catalog, where it is indexed for easy discovery. In addition, data professionals can contribute their knowledge in order to enrich the source metadata.

Azure Event Hubs : An event processing service that can ingest millions of events per second and make them available for storage and analysis. The service can log events in near real time and accept data from a wide range of sources. Event Hubs uses technologies that support low latency and high availability, while providing flexible throttling, authentication, and scalability.

Microsoft Envision | Advantage YOU: Be more, do more, with Infosys and Microsoft on your side


For more information about Event Hubs, refer to the Simple-Talk article Azure Event Hubs. In the meantime, here’s a quick overview of the analytic components included in the CAS package:

Azure Machine Learning : A service for building, deploying, and sharing predictive analytic solutions. The service runs predictive models that learn from existing data, making it possible to forecast future behavior and trends. Machine Learning also provides the tools necessary for testing and managing the models as well as deploying them as web services.
Azure Data Lake Analytics : A distributed service for analyzing data of any size, including what is in Data Lake Store. Data Lake Analytics is built on Apache YARN, an application management framework for processing data in Hadoop clusters. Data Lake Analytics also supports U-SQL, a new language that Microsoft developed for writing scalable, distributed queries that analyze data.

Azure HDInsight : A fully managed Hadoop cluster service that supports a wide range of analytic engines, including Spark, Storm, and HBase. Microsoft has updated the service to take advantage of Data Lake Store and to maximize security, scalability, and throughput.
Azure Stream Analytics : A service that supports complex event processing over streaming data. Stream Analytics can handle millions of events per second from a variety of sources, while correlating them across multiple streams. It can also ingest events in real-time, whether from one data stream or multiple streams.
I’ve already mentioned how Data Lake Analytics and HDInsight are part of Azure Data Lake, and I’ve pointed you to a related article. If you want to learn more about Stream Analytics, check out the Simple-Talk article Microsoft Azure Stream Analytics.

Azure Stream Analytics

Cortana Analytics Gallery

Another interesting component of the CAS package is the Cortana Analytics Gallery, formerly the Azure Machine Learning Gallery. The gallery provides an online environment for data scientists and developers to share their solutions, particularly those related to machine learning. Microsoft also publishes its own solutions to the site for participants to consume. Cortana Analytics gallery

The Cortana Analytics Gallery is divided into the following six sections.

Solution Templates : Templates based on industry-specific partner solutions. Currently, the category includes only the Vehicle Telemetry Analytics solution, published by Microsoft this past December. The solution demonstrates how those in the automobile industry can gain real-time and predictive insights into vehicle health and driving habits.
Experiments : Predictive analytic experiments contributed by Microsoft and those in the data science community. The experiments demonstrate advanced machine learning techniques and can be used as a starting point for developing your own solutions. For example, the Telco Customer Churn experiment uses classification algorithms to predict whether a customer will churn.
Machine Learning APIs : APIs that can access operationalized predictive analytic solutions. Some of the APIs are reference within the “Perceptual intelligence” section listed in the table above. For example, the Face APIs were published by Microsoft and are part of Microsoft Project Oxford. They provide state-of-the-art algorithms for processing face images.
Notebooks : A collection of Jupyter notebooks. The notebooks are integrated within Machine Learning Studio and serve as web applications for running code, visualizing data, and trying out ideas. For example, the notebook Topic Discovery in Twitter Tweets demonstrates how a Jupyter notebook can be used for mining Twitter text.
Tutorials : Tutorials on how to use Cortana Analytics to solve real-world problems. For example, the iPhone app for RRS tutorial describes how to create an iOS app that can consume an Azure ML RRS API using the Xamarin development software that ships with Visual Studio.
Collections : A site for grouping together experiments, templates, APIs, or other items within the Cortana Analytics Gallery.
Although Microsoft has changed the name of the gallery to make it more CAS-friendly, much of the content still focuses on the Machine Learning service. Even so, the gallery could prove to be a valuable resource for organizations jumping aboard the CAS train, particularly once the gallery has gained more momentum.

Cortana Analytics Workshop: The "Big Data" of the Cortana Analytics Suite, Part 1


Cortana Analytics Workshop: The "Big Data" of the Cortana Analytics Suite, Part 2

More Information:










Share:

0 reacties:

Post a Comment