• IBM Consulting

    DBA Consulting can help you with IBM BI and Web related work. Also IBM Linux is our portfolio.

  • Oracle Consulting

    For Oracle related consulting and Database work and support and Migration call DBA Consulting.

  • Novell/RedHat Consulting

    For all Novell Suse Linux and SAP on Suse Linux questions releated to OS and BI solutions. And offcourse also for the great RedHat products like RedHat Enterprise Server and JBoss middelware and BI on RedHat.

  • Microsoft Consulting

    For Microsoft Server 2012 onwards, Microsoft Client Windows 7 and higher, Microsoft Cloud Services (Azure,Office 365, etc.) related consulting services.

  • Citrix Consulting

    Citrix VDI in a box, Desktop Vertualizations and Citrix Netscaler security.

  • Web Development

    Web Development (Static Websites, CMS Websites (Drupal 7/8, WordPress, Joomla, Responsive Websites and Adaptive Websites).

24 October 2017

SQL Server 2017 on Linux



SQL Server 2017 on Linux

Microsoft has heard from you, our customers, that your data estate gets bigger, more complicated, and more diverse every year. You need solutions that work across platforms, whether on-premises or in the cloud, and that meet your data workloads where they are. Embracing this choice, earlier today we announced the general availability of SQL Server 2017 on Linux, Windows, and Docker on October 2, 2017.



Today, Microsoft and Red Hat are delivering on choice by announcing the availability of Microsoft SQL Server 2017 on Red Hat Enterprise Linux, the world’s leading enterprise Linux platform. As Microsoft’s reference Linux platform for SQL Server, Red Hat Enterprise Linux extends the enterprise database and analytics capabilities of SQL Server by delivering it on the industry-leading platform for performance, security features, stability, reliability, and manageability.

Customers will be able to bring the performance and security features of SQL Server to Linux workloads. SQL Server 2017 on Red Hat Enterprise Linux delivers mission-critical OLTP database capabilities and enterprise data warehousing with in-memory technology across workloads. SQL Server 2017 embraces developers by delivering choice in language and platform, with container support that seamlessly facilitates DevOps scenarios. The new release of SQL Server delivers all of this, built-in. And, it runs wherever you want, whether in your datacenter, in Azure virtual machines, or in containers running on Red Hat OpenShift Container Platform!



Also starting October 2nd until June 30th, 2018, we are launching a SQL Server on Red Hat Enterprise Linux offer to help with upgrades and migrations. This offer provides up to 30% off SQL Server 2017 through an annual subscription. When customers purchase a new Red Hat Enterprise Linux subscription to support their SQL Server, they will be eligible for another 30% off their Red Hat Enterprise Linux subscription price.

In addition to discounts on SQL Server and Red Hat Enterprise Linux, all of this is backed by integrated support from Microsoft and Red Hat.

Bootcamp 2017 - SQL Server on Linux


 SQL Server 2017 is generally available for purchase and download! The new release is available right now for evaluation or purchase through the Microsoft Store, and will be available to Volume Licensing customers later today. Customers now have the flexibility for the first time ever to run industry-leading SQL Server on their choice of Linux, Docker Enterprise Edition-certified containers and, of course, Windows Server. It’s a stride forward for our modern and hybrid data platform across on-premises and cloud.

Everything you need to know about SQL Server 2017


In the 18 months since announcing our intent to bring SQL Server to Linux, we’ve been focused on making SQL Server perform and scale to the industry-leading levels customers expect from SQL Server, making SQL Server feel familiar yet native to Linux, and ensuring compatibility between SQL Server on Windows and Linux. With all the enterprise database features you rely on, from Active Directory authentication, to encryption, to Always On availability groups, to record-breaking performance, SQL Server is at parity on Windows and Linux. We have also brought SQL Server Integration Services to Linux so that you can perform data integration just like on Windows. SQL Server 2017 supports Red Hat Enterprise Linux, SUSE Linux Enterprise Server, and Ubuntu.

There are a number of new features for SQL Server that we think make this the best release ever. Here are just a few:

  • Container support seamlessly facilitates your development and DevOps scenarios by enabling you to quickly spin up SQL Server containers and get rid of them when you are finished. SQL Server supports Docker Enterprise Edition, Kubernetes and OpenShift container platforms.
  • AI with R and Python analytics enables you to build intelligent apps using scalable, GPU-accelerated, parallelized R and now Python analytics running in the database.
  • Graph data analysis will enable customers to use graph data storage and query language extensions for graph-native query syntax in order to discover new kinds of relationships in highly interconnected data.
  • Adaptive Query Processing is a new family of features in SQL Server that bring intelligence to database performance. For example, Adaptive Memory Grants in SQL Server track and learn from how much memory is used by a given query to right-size memory grants.
  • Automatic Plan Correction ensures continuous performance by finding and fixing performance regressions.



Above and beyond these top-line features, there are more enhancements that you haven’t heard as much about, but we hope will truly delight you:


  • Resumable online index rebuild lets you stop and start index maintenance. This gives you the ability to optimize index performance by re-indexing more frequently – without having to wait for a long maintenance window. It also means you can pick up right where you left off in the event of a disruption to database service.
  • LOB compression in columnstore indexes. Previously, it was difficult to include data which contained LOBs in a columnstore index due to size. Now those LOBs can be compressed, making LOBs easier to work with and broadening the applicability of the columnstore feature.
  • Clusterless availability groups enable you to scale out reads by building an Always On availability group without having to use an underlying cluster.
  • Continued improvement to key performance features such as columnstore, in-memory OLTP, and the query optimizer to drive new record-setting performance. We’ll share some even more exciting perf and scale numbers soon!
  • Native scoring in T-SQL lets you score operational data using advanced analytics in near real-time because you don’t have to load the Machine Learning libraries to access your model.
  • SQL Server Integration Services (SSIS) scale-out enables you to speed package execution performance by distributing execution to multiple machines. These packages are executed in parallel, in a scale-out mode.
What’s new in SQL Server 2017





Many enhancements were made to SQL Server Analysis Services including:
  • Modern “get data” experience with a number of new connectors like Oracle, MySQL, Sybase, Teradata, and more to come. New transformations enable mashing up of the data being ingested into tabular models.
  • Object-level security for tables and columns.
  • Detail rows and ragged hierarchies support, enabling additional drill-down capabilities for your tabular models.
Enhancements were made to SQL Server Reporting Services as well, including:
  • Lightweight installer with zero impact on your SQL Server databases or other SQL Server features.
  • REST API for programmatic access to reports, KPIs, data sources, and more.
  • Report comments, enabling users to engage in discussion about reports.



In addition to the ability to upgrade existing SQL Server to 2017, there are a few more benefits to renewing your software assurance:


  • Machine Learning Server for Hadoop, formerly R Server, brings R and Python based, scalable analytics to Hadoop and Spark environments, and it is now available to SQL Server Enterprise edition customers as a Software Assurance benefit.
  • SQL Server Enterprise Edition Software Assurance benefits also enable you to run Power BI Report Server. Power BI Report Server enables self-service BI and enterprise reporting, all in one solution by allowing you to manage your SQL Server Reporting Services (SSRS) reports alongside your Power BI reports. Power BI Report Server is also included with the purchase of Power BI Premium.
  • Lastly, but importantly, we are also modernizing how we service SQL Server. Please see our release management blog for all the details on what to expect for servicing SQL Server 2017 and beyond.

Microsoft will continue to invest in SQL Server 2017 and cloud-first development model, to ensure that the pace of innovation stays fast.

SQL Server 2017 sets the standard when it comes to speed and performance. Based on the incredible work of SQL Server 2016 (See the blog series It Just Runs Faster), SQL Server 2017 is fast: built-in, simple, and online. Maybe you caught my presentation at Microsoft Ignite where I demonstrated 1 million transactions per minute on my laptop using the popular tool HammerDB¹ by simply installing SQL Server out of the box with no configuration changes (with the HammerDB client and SQL Server on the same machine!)

SQL Server 2017 on Linux Introduction


Consider for a minute all the built-in capabilities that power the speed of SQL Server. From a SQLOS scheduling engine that minimizes OS context switches to read-ahead scanning to automatic scaling as you add NUMA and CPUs. And we parallelize everything! From queries to indexes to statistics to backups to recovery to background threads like LogWriter. We partition and parallelize our engine to scale from your laptop to the biggest servers in the world.

Like the enhancements we made as described in It Just Runs Faster, in SQL Server 2016, we are always looking to tune our engine for speed, all based on customer experiences. Take, for example, indirect checkpoint, which is designed to provide a more predictable recovery time for a database. We boosted scalability of this feature based on customer feedback. We also made scalability improvements for parallel scanning and consistency check performance. No knobs required. Just built-in for speed.

One of the coolest performance aspects to built-in speed is online operations. We know you need to perform other maintenance tasks than just run queries, but keep your application up and running, so we support online backups, consistency checks, and index rebuilds. SQL Server 2017 enhances this functionality with resumable online index builds allowing you to pause an index build and resume it at any time (even after a failure).

Microsoft SQL Server 2017 Deep Dive


SQL Server 2017 is faster than you think. SQL Server 2017 was designed from the beginning to run fast on popular Linux distributions such as Red Hat Enterprise Linux, SUSE Linux Enterprise, and Ubuntu whether that is on your server or a Docker Container. Don’t believe it? Check out our world record 1TB TPC-H benchmark result (non-clustered) for SQL Server on Red Hat Enterprise Linux. Even though this is our first release on Linux, we know how to configure and run SQL Server on Linux for maximum speed. Read our best practices guide for performance settings on Linux in our documentation. We know it performs well because our customers tell us. Read the amazing story of dv01 and how SQL Server on Linux exceeded their performance expectations as they migrated from PostgreSQL

SQL Server 2017 Deep Dive - @Ignite 2017


One of the key technologies to achieve a result like this is columnstore indexes. This is one of the most powerful features of SQL Server for high-speed analytic queries and large databases. Columnstore indexes boost performance by organizing columnar data in a new way than traditional indexes, compressing data to reduce memory and disk footprint, filtering scans automatically through rowgroup elimination and processing queries in batches. SQL Server runs at warp speed for data warehouses and columnstore is the fuel. At Microsoft Ignite, I demonstrated how columnstore indexes can make PowerBI with Direct Query against SQL Server faster handling the self-service, ad-hoc nature of PowerBI queries.

Microsoft Ignite 2017 - SQL Server on Kubernetes, Swarm, and Open Shift


SQL Server also excels at transaction processing, the heart of many top enterprise workloads. Got RAM? Not only does columnstore use in-memory technologies to achieve speed, but our In-Memory OLTP feature focuses on optimized access to memory-optimized tables. This feature is named OLTP, but it can be so much more. ETL staging tables, IoT workloads, table types (no more tempdb!), and “caching” tables. One of our customers was able to get a throughput of 1.2M batch requests/sec using SCHEMA_ONLY memory-optimized tables. To really boost transaction processing, also consider using SQL Server’s support for Persistent Memory (NVDIMM-N) and our optimization for transaction log (get ready for WRITELOG waits = 0!) performance. SQL Server 2017 supports any Persistent Memory technology supported on Windows Server 2016 and later releases.

Many customers I talk to have great performance when they first deploy SQL Server and their application. Keeping SQL Server fast and tuned is more of the challenge. SQL Server 2017 comes with features to keep you fast and tuned automatically and adaptively. Our Query Processing engine has all types of capabilities to create and build query plans to maximize the performance of your queries. We have created a new feature family in SQL Server 2017 to make it smarter, called Adaptive Query Processing. Imagine running a query that is not quite the speed you expect because of insufficient memory grants (which is a thorn in the side of many SQL Server users, as it can lead to a spill to tempdb). With Adaptive Query Processing, future executions of this query will have a corrected calculated memory grant avoiding the spill, all without requiring a recompilation of the query plan. Adaptive Query Processing handles other scenarios such as adaptive joins and interleaved execution of Table Valued Functions.

Choosing technologies for a big data solution in the cloud


Another way to keep you tuned is the amazing feature we added in SQL Server 2016 called Query Store. Query Store provides built-in capabilities to track and analyze query performance all stored in your database. For SQL Server 2017, we made tuning adjustments to Query Store to make it more efficient based on learnings in our Azure SQL Database service where Query Store is enabled for millions of databases. We added wait statistics so now you have an end-to-end picture of query performance. Perhaps though the most compelling enhancement in SQL Server 2017 is Automatic Tuning. Parameter Sniffing got you down? Automatic Tuning uses Query Store to detect query plan regressions and automatically forces a previous plan that used to run fast. What I love about this feature is that even if you don’t have it turned on, you can see recommendations it has detected about plan regressions. Then you can either manually force plans that you feel have regressed or turn on the feature to have SQL Server do it for you.

Introduction to PolyBase


SQL Server 2017 is the fastest database everywhere you need it. Whether it is your laptop, in your private cloud, or in our Azure public cloud infrastructure. Whether it is running on Linux, Windows, or Docker Containers, we have the speed to power any workload your application needs.

As I mentioned above, back in April, we announced our world record TPC-H 1TB data warehousing workload (non-clustered) for SQL Server 2017 running on a HPE ProLiant DL380 Gen9 using RedHat Enterprise Linux².

Perhaps you missed the announcement in June of 2017, of a new world record TPC-E benchmark result³ on SQL Server 2017 on Windows Server 2016 running on a Lenovo ThinkSystem SR650 continuing to demonstrate our leadership in database performance. This benchmark running on a 2 socket system using Intel’s Xeon Scalable Processors has set a new standard for price and performance, becoming the first TPC-E benchmark result ever to be under $100/tpsE.

We continued to show our proven speed for analytics by announcing in July of 2017 a new TPC-H 10TB (non-clustered) world record benchmark result4 of 1,336,109 QppH on Windows Server 2016 using a Lenovo ThinkSystem SR950 system with 6TB RAM and 224 logical CPUs.

While benchmarks can show the true speed of SQL Server, we believe it can perform well with your workload and maximize the computing power of your server. Perhaps you caught the session at Ignite where my colleague Travis Wright showed how we can scan a 180 Billion row table (from a 30TB database) in our labs in under 20 seconds powering 480 CPUs to 100% capacity. And if you don’t believe SQL Server is deployed in some of the biggest installations and servers in the world, I recently polled some of our field engineers, SQL Customer Advisor Team, and MVPs asking them for their largest SQL Server deployments. Over 30 people responded, and the average footprint of these installations was 3TB+ RAM on machines with 128 physical cores. Keep in mind that SQL Server on can theoretically scale to 24TB RAM on Windows and 64TB on Linux. And it supports the maximum CPUs of those systems (64 sockets with unlimited cores on Windows and 5120 logical CPUs on Linux). Look for more practical and fun demonstrations of the speed of SQL Server in the future.

Microsoft cloud big data strategy


It could be that you are consolidating your deployments and want to run SQL Server using Azure Virtual Machine, but not sure if the capacity is there for your performance needs. Consider that Azure Virtual machine has the new M-Series, which supports up to 128 vCPUs, 2TB RAM, and 64 Data Disks with a capacity of 160,000 IOPS. It could be that in your environment you want to scale out your read workload with Availability Group secondary replicas but don’t want to invest in Failover Clustering. SQL Server 2017 introduces the capability of read-scale availability groups without clustering supported both on Windows and Linux. Two other very nice performance features new to SQL Server 2017 are SSIS Scale Out, for those with data loading needs, and native scoring, which integrates machine learning algorithms into the SQL Server engine for maximum performance.

Microsoft Technologies for Data Science 201612


SQL Server 2017 brings to the database market a unique set of features and speed. A database engine that is fast, built-in with the power to scale, and even faster when taking advantage of technologies like columnstore Indexes and In-Memory OLTP. An engine that provides automation and adapts to keep you fast and tuned. And the fastest database everywhere you need it.

Machine learning services with SQL Server 2017

More Information:

https://docs.microsoft.com/en-us/sql/sql-server/sql-server-2017-release-notes

http://www.databasejournal.com/features/mssql/slideshows/9-new-features-with-sql-server-2017.html

https://www.microsoft.com/en-us/sql-server/sql-server-2017

https://myignite.microsoft.com/sessions

https://blogs.technet.microsoft.com/dataplatforminsider/2017/10/02/sql-server-2017-on-windows-linux-and-docker-is-now-generally-available/

https://blogs.msdn.microsoft.com/bobsql/

http://www.hammerdb.com

http://www.hammerdb.com/benchmarks.html

https://blogs.msdn.microsoft.com/sqlserverstorageengine/

https://blogs.technet.microsoft.com/dataplatforminsider/2017/09/28/enhancing-query-performance-with-adaptive-query-processing-in-sql-server-2017/

https://blogs.technet.microsoft.com/dataplatforminsider/2017/09/29/view-on-demand-microsoft-data-platform-sql-server-2017-and-azure-data-services/

https://blogs.technet.microsoft.com/dataplatforminsider/2017/10/10/whats-new-in-sql-server-management-studio-17-3/

https://redmondmag.com/articles/2017/09/25/microsoft-launches-sql-server-2017.aspx

https://redmondmag.com/articles/2017/04/19/sql-server-2017-preview.aspx

https://arstechnica.com/gadgets/2017/09/microsoft-ignite-2017-azure-sql/











26 September 2017

Oracle Sparc M8 and Oracle Advanced Analytics



Oracle SPARC M8 released with 32 cores 256 threads 5.0GHz

Oracle announced its eighth generation SPARC platform, delivering new levels of security capabilities, performance, and availability for critical customer workloads. Powered by the new SPARC M8 microprocessor, new Oracle systems and IaaS deliver a modern enterprise platform, including proven Software in Silicon with new v2 advancements, enabling customers to cost-effectively deploy their most critical business applications and scale-out application environments with extreme performance both on-premises and in Oracle Cloud.

Oracle’s Advanced Analytics & Machine Learning 12.2c New Features & Road Map; Bigger, Better, Faster, More!



SPARC M8 processor-based systems, including the Oracle SuperCluster M8 engineered systems and SPARC T8 and M8 servers, are designed to seamlessly integrate with existing infrastructures and include fully integrated virtualization and management for private cloud. All existing commercial and custom applications will run on SPARC M8 systems unchanged with new levels of performance, security capabilities, and availability. The SPARC M8 processor with Software in Silicon v2 extends the industry's first Silicon Secured Memory, which provides always-on hardware-based memory protection for advanced intrusion protection and end-to-end encryption and Data Analytics Accelerators (DAX) with open API's for breakthrough performance and efficiency running Database analytics and Java streams processing. Oracle Cloud SPARC Dedicated Compute service will also be updated with the SPARC M8 processor.

Spark SQL: Another 16x Faster After Tungsten: Spark Summit East talk by Brad Carlile



"Oracle has long been a pioneer in engineering software and hardware together to secure high-performance infrastructure for any workload of any size," said Edward Screven, chief corporate architect, Oracle. "SPARC was already the fastest, most secure processor in the world for running Oracle Database and Java. SPARC M8 extends that lead even further."



The SPARC M8 processor offers security enhancements delivering 2x faster encryption and 2x faster hashing than x86 and 2x faster than SPARC M7 microprocessors. The SPARC M8 processor's unique design also provides always-on security by default and built-in protection of in-memory data structures from hacks and programming errors.



SPARC M8's silicon innovation provides new levels of performance and efficiency across all workloads, including: 
  • Database: Engineered to run Oracle Database faster than any other microprocessor, SPARC M8 delivers 2x faster OLTP performance per core than x86 and 1.4x faster than M7 microprocessors, as well as up to 7x faster database analytics than x86.
  • Java: SPARC M8 delivers 2x better Java performance than x86 and 1.3x better than M7 microprocessors.  DAX v2 produces 8x more efficient Java streams processing, improving overall application performance.
  • In Memory Analytics: Innovative new processor delivers 7x Queries per Minute (QPM)/core than x86 for database analytics.
Oracle is committed to delivering the latest in SPARC and Solaris technologies and servers to its global customers. Oracle's long history of binary compatibility across processor generations continues with M8, providing an upgrade path for customers when they are ready. Oracle has also publicly committed to supporting Solaris until at least 2034.

Oracle Sparc M8 is available for:

  • Oracle SPARC M8
  • Oracle SPARC T8-1 server
  • Oracle SPARC T8-2 server
  • Oracle SPARC T8-4 server
  • Oracle SPARC M8-8 server
  • Oracle SuperCluster M8 engineered system

More information in: Oracle SPARC M8 Launch Webcast:  http://www.oracle.com/us/corporate/events/next-gen-secure-infrastructure-platform/index.html

About Oracle 

The Oracle Cloud offers complete SaaS application suites for ERP, HCM and CX, plus best-in-class database Platform as a Service (PaaS) and Infrastructure as a Service (IaaS) from data centers throughout the Americas, Europe and Asia. For more information about Oracle (NYSE: ORCL), please visit us at oracle.com.

Big data analytics using oracle advanced analytics and big data sql



The Oracle SPARC M8 is now out and is a monster of a chip. Each SPARC M8 processor supports up to 32 cores and 64MB L3 cache. Each core can handle 8 threads for up to 256 threads. Compare this to the AMD EPYC 7601, the world’s only 32 core x86 processor as of this writing, which handles 64 threads and also has 64MB L3 cache. The cores can also clock up to 5.0GHz faster than current x86 high-core count server chip designs from Intel and AMD. That is quite astounding given the SPARC M8 is still using 20nm process technology.

Beyond simple the simple core specs, there is much more going on. Oracle has specific accelerators for cryptography, JAVA performance, database performance and ETC. For example, there are 32 on-chip Data Analytics Accelerator (DAX) engines. DAX engines offload query processing and perform real-time data decompression. Oracle’s software business for the Oracle Database line is still strong and these capabilities are what is often referred to as “SQL in Silicon.” Oracle claims that Oracle Database 12c is up to 7 times faster by using M8 with DAX than competing CPUs. That is a big deal for software licensing costs. Another interesting feature is the inline decompression feature allows decompression of data stored in memory with no claimed performance penalty.

Oracle SPARC M8 Processor Key Specifications

Here are the key specs for the new Oracle SPARC CPUs:


  • 32 SPARC V9 cores, maximum frequency: 5.0 GHz
  • Up to 256 hardware threads per processor; each core supports up to 8 threads
  • Total of 64 MB L3 cache per processor, 16-way set-associative and inclusive of all inner caches
  • 128 KB L2 data cache per core; 256 KB L2 instruction cache shared among four cores
  • 32 KB L1 instruction cache and 16 KB L1 data cache per core
  • Quad-issue, out-of-order integer execution pipelines, one floating-point unit, and integrated cryptographic stream processing per core
  • Sophisticated branch predictor and hardware data prefetcher
  • 32 second-generation DAX engines; 8 DAX units per processor with four pipelines per DAX unit
  • Encryption instruction accelerators in each core with direct support for 16 industry-standard cryptographic algorithms plus random-number generation: AES, Camellia, CRC32c, DES, 3DES, DH, DSA, ECC, MD5, RSA, SHA-1, SHA-3, SHA-224, SHA-256, SHA-384, and SHA-512
  • 20 nm process technology
  • Open Oracle Solaris APIs available for software developers to leverage the Silicon Secured Memory and DAX technologies in the SPARC M8 processor
  • On Solaris Support Until 2034


In the official Oracle SPARC M8 release, Oracle has a note that is a clear nod to its Organizationals changes (we mentioned in a recent Oracle server release.)

Oracle is committed to delivering the latest in SPARC and Solaris technologies and servers to its global customers. Oracle’s long history of binary compatibility across processor generations continues with M8, providing an upgrade path for customers when they are ready. Oracle has also publicly committed to supporting Solaris until at least 2034.

Oracle is clearly hearing from its customers about the mass layoffs of Solaris engineering teams.

New Oracle SPARC M8 Systems

There are five new SPARC V9 systems are available from Oracle today:

  • Oracle SPARC T8-1 server
  • Oracle SPARC T8-2 server
  • Oracle SPARC T8-4 server
  • Oracle SPARC M8-8 server
  • Oracle SuperCluster M8 engineered system

The Evolution and Future of Analytics

We live in a world where things around us are ever changing.



Measurement metrics are just in time, predictive and need a lot of augmented intelligence; however, we're developing more complex mind analytics when it comes to buying patterns.

This new type of analytics can give us insight into how the customer feels and what he or she experiences.

Oracle's Machine Learning & Advanced Analytics 12.2 & Oracle Data Miner 4.2 New Features


Thus, the availability of smart information will emerge.

In the future, you may walk into a store and find one or all of the below, which can be built as solutions:

a) A robot welcoming you and taking over to interact with you using connected back end and analytics.

b) Natural language or human analytics that can automatically read your mood to ultimately improve customer satisfaction.

c) Historical data about you as a customer to help up sell or cross sell products based on your interests.

d) Automatic analysis about what you're doing to bring near real-time context of data; this will enable the retailer to build a mobile based intuitive presence or no billing architecture.

e) A personal assistant model to better serve you as a customer, empowering retailers to provide solutions to unsure customers.

f) IN product or things analytics to provide information about the product that makes things intelligent through RFID, intelligent tagging, sensors etc.

g) Discounts/coupons based on mixing historical buying patterns; post purchase analytics.

h) Interactive dashboards that make augmented decisions about a few areas based on reviews; this would take expert reviews, phone calls, product management and more into account.

i) A store platform of grammar, syntax, semantics and data science grammar to create recurring patterns, challenges and build new solutions which are continuous in nature.

Based on the above, let's dive into different types of analytics available on the market. We'll look at how they will blend and intersect to develop more augmented applications for the future.

Insights into Real-world Data Management Challenges





1) Historical Analytics

This is the traditional analytics of business intelligence focused on analyzing stored data and reporting. We would build repositories and create analyses and dashboards for historical data. Solutions would include Oracle Business Intelligence.

2) Current Analytics 

Here the analytics is measurement over current process. For example, we would measure the effectiveness of a process as it happens (business activity monitoring) using a stream that processes arriving data and analyzes it in real-time.

3) Enterprise Performance Management

Here the objective is to focus on projections/what-if analysis with the current data and make projections for the future. An example would be a Hyperion or an EPM based solution which could help derive and plan reporting as projections. EPM today is also available as a cloud service.

4) Predictive Analytics

With the Big Data market growing, and with unstructured data adding parameters of velocity, variety and volume, the data world is moving on to more predictive analytics, with a blended mix of data. There is one world of data in the hadoop world and another in the classical data warehouse world. We can mix and match and do Big Data analytics.

Predictive analytics is more of a compass-like decision making with data analysis patterns. Oracle has an end-to-end Big Data solution from DW, Hadoop and analytics that can help develop predictive solutions.

MSE 2017 - Customer Voted Session: Rocketing Your Knowledge Management Success with Analytics


5) Prescriptive Analytics

To extend the predictive analytics, we would also develop systems to make decisions once we have the prediction; i.e. sending emails and connecting systems as the patterns are detected. This is the basics of building more heuristics systems to make decisions about arrived patterns.

6) Machine Analytics 

Every device and machine is going to generate data. Machine analytics is a blended form of data that can be embedded into the standard source to enhance and improve the overall data pattern. Oracle provides IOT CS as a solution to connect, analyze and integrate data from various machines and enrich new applications like  ERP, CRM and more.

Oracle Analytics and Big Data: Unleash the Value



7) AI Based Analytics

AI or deep learning is the next gen analytics pattern where we can train the systems or any entity to think and then embed the analytics pattern in the solution.

8) IORT / Robotics Analytics

With Robots / Bots and personal assistant complementing solutions, there are a lot of patterns of thinking and execution distributed to multiple systems. IORT or robotics analytics is a new branch that will focus on how we can analyze the pattern from semi thinking devices.

9) Data Science as Service 

A new branch where the analysis goes deeper in terms of algorithms and storage and is also more domain-driven. Even though data science is used as one branch in analytics, you will see a lot of analytics development. Data scientists who specialize in identifying patterns will go a long way to build patterns that are more replicable.

10) Integrated Analytics

In the future, we can form an integrated view of the above. This could be ONE IDE and you would derive patterns based on business need and use case. Today, we have a fragmented set of tools to manage analytics and it would slowly get integrated into one view.
Oracle has solution at different levels; most of them are also available as a cloud service (Software as a Service, Platform as a Service).

MSE 2017 - Advanced Analytics for Developers


It's imperative to build the right mix of solutions for the right problem and integrate these solutions.

  • Historical perspective you would use --> Business Intelligence 
  • Current processing  -->  Streaming (event processing) and Business Activity Monitoring
  • Enterprise performance management  --> Hyperion
  • Heterogeneous source of data and also large analysis of data --> Big Data Solution
  • Predictive and Prescriptive analytics --> R language and Advanced Analytics
  • Machine related --> IOT Solutions and Cloud Service

Oracle Architectural Elements Enabling R for Big Data


Oracle University provides competency solutions for all the above and empowers you with skill development and well-respected certifications that validate your expertise:


  • Big Data Analytics training
  • BI Data Analytics training
  • Hyperion training
  • Cloud PAAS Platform for Analytics and BI training



More Information:

http://www.oracle.com/technetwork/database/options/advanced-analytics/overview/index.html

http://www.oracle.com/technetwork/database/database-technologies/bdc/r-advanalytics-for-hadoop/overview/index.html

http://www.oracle.com/technetwork/database/options/advanced-analytics/r-enterprise/learnmore/user2017-archelements-hornick-3850449.pdf

https://blogs.oracle.com/bigdata/oracles-machine-learning-and-advanced-analytics-122-and-oracle-data-miner-42-new-features

https://blogs.oracle.com/bigdata/announcing-oracle-data-integrator-for-big-data

http://www.oracle.com/us/corporate/events/next-gen-secure-infrastructure-platform/index.html

https://www.nextplatform.com/2017/09/18/m8-last-hurrah-oracle-sparc/

https://blogs.oracle.com/datamining/evaluating-oracle-data-mining-has-never-been-easier-evaluation-kit-available-•-updated-for-oracle-database-122c-sqldev-42

https://blogs.oracle.com/datamining/

https://www.oracle.com/corporate/pressrelease/oracle-forrester-analytics-cloud-092117.html

https://blogs.oracle.com/datawarehousing/compendium/page/2

http://www.odtug.com/p/bl/et/blogaid=713

Oracle Visual Analytics

http://www.prnewswire.com/news-releases/english-releases/oracles-new-sparc-systems-deliver-2-7x-better-performance-security-capabilities-and-efficiency-than-intel-based-systems-300521018.html

https://blogs.oracle.com/datamining/oracle-biwa17-the-big-data-analytics-spatial-cloud-iot-everything-cool-oracle-user-conference-2017

http://www.vlamis.com/papers2017/

https://blogs.oracle.com/datawarehousing/announcing-oracle-advanced-analytics



22 August 2017

Oracle Database 12c Release 2

Oracle Database 12c Release 2 (12.2), is now available everywhere 

Ask Tom Answer Team (Connor McDonald and Chris Saxon) on Oracle Database 12c Release 2 New Features



Oracle Database 12.2c Architecture Diagram
http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/12c/r1/poster/OUTPUT_poster/pdf/Database%20Architecture.pdf


The latest generation of the world's most popular database, Oracle Database 12c Release 2 (12.2), is now available everywhere - in the Cloud, with Oracle Cloud at Customer, and on-premises.  This latest release provides organizations of all sizes with access to the world’s fastest, most scalable and reliable database technology in a cost-effective, hybrid Cloud environment. 12.2 also includes a series of innovations that helps customers easily transform to the Cloud while preserving their investments in Oracle Database technologies, skills and resources.

Oracle RAC 12c Release 2 New Features

Database Security - Comprehensive Defense in Depth

Partner Webcast – Oracle Identity Cloud Service: Introducing Secure, On-Demand Identity Management



Oracle Database 12c provides multi-layered security including controls to evaluate risks, prevent unauthorized data disclosure, detect and report on database activities and enforce data access controls in the database with data-driven security. Oracle Database 12c Release 2 (12.2), now available in the Cloud and on-premises, introduces new capabilities such as on-line and off-line tablespace encryption and database privilege analysis. Combined with Oracle Key Vault and Oracle Audit Vault and Database Firewall, Oracle Database 12c provides unprecedented defense-in-depth capabilities to help organizations address existing and emerging security and compliance requirements.

Partner Webcast – Enabling Oracle Database High Availability and Disaster Recovery with Oracle Cloud


Database Cloud Services

Oracle Cloud provides several Oracle Cloud Service deployment choices. These choices allow you to start at the cost and capability level suitable to your use case and then gives you the flexibility to adapt as your requirements change over time. Choices include: single schemas, dedicated pluggable databases, virtualized databases, bare metal databases and databases running on world class engineered infrastructure.

The Oracle Exadata Cloud Service offers the largest most business-critical database workloads a place to run in Oracle Cloud. With all the infrastructure components including hardware, networking, storage, database and virtualization in place, access to secured, highly available and powerful performance is easily provisioned in a few clicks. Exadata Cloud Service is engineered to support OLTP, Data Warehouse / Real-Time Analytic and Mixed database workloads at any scale. With this service, you maintain control of your database while Oracle manages the hardware, storage and networking infrastructure letting you focus on growing your business.



https://cloud.oracle.com/database

Oracle Database Exadata Cloud Machine delivers the world’s most advanced database cloud to customers who require their databases to be located on-premises. Exadata Cloud Machine uniquely combines the world’s #1 database technology and Exadata, the most powerful database platform, with the simplicity, agility and elasticity of a cloud-based deployment. It is identical to Oracle’s Exadata public cloud service, but located in customers’ own data centers and managed by Oracle Cloud Experts. Every Oracle Database and Exadata feature and option is included with the Exadata Cloud Machine subscription, ensuring highest performance, best availability, most effective security and simplest management. Databases deployed on Exadata Cloud Machine are 100% compatible with existing on-premises databases, or databases that are deployed in Oracle’s public cloud. Exadata Cloud Machine is ideal for customers desiring cloud benefits but who cannot move their databases to the public cloud due to sovereignty laws, industry regulations, corporate policies, or organizations that find it impractical to move databases away from other tightly coupled on-premises IT infrastructure.

Oracle Database 12c Release 2 Sharded Database Overview and Install (Part 1)


Oracle Sharding Part 2


Oracle Sharding Part 3


Oracle Sharding with Suresh Gandhi

Overview of Oracle‘s Big Data Management System

As today's enterprises embrace big data, their information architectures are evolving. The new information architecture in the big data era embraces emerging technologies such as Hadoop, but at the same time leverages the core strengths of previous data warehouse architectures.

Partner Webcast – Oracle Ravello Cloud Service: Easy Deploying of Big Data VM on Cloud



The data warehouse, built upon Oracle Database 12c Release 2 and Exadata, will continue to be the primary analytic database for storing core transactional data: financial records, customer data, point- of-sale data and so forth (see Key Data Warehousing and Big Data Capabilities for more information).

However, the data warehouse will be augmented by a big-data system (built upon Oracle Big Data Appliance), which functions as a ‘data reservoir’. This will be the repository for the new sources of large volumes of data: machine-generated log files, social-media data, and videos and images -- as well as a repository for more granular transactional data or older transactional data which is not stored in the data warehouse.

Data flows between the big data system and the data warehouse to create a unified foundation: the Oracle Big Data Management System.

The transition from the Enterprise Data Warehouse centric architecture to the Big Data Management System - both on-premise, on the Cloud, or in hybrid Cloud systems - is going to revolutionize any companies information management architecture. Oracle's Statement of Direction outlines Oracle's vision for delivering innovative new technologies for building the information architecture of tomorrow.

Partner Webcast – Docker Agility in Cloud: Introducing Oracle Container Cloud Service





Big data is in many ways an evolution of data warehousing. To be sure, there are new technologies used for big data, such as Hadoop and NoSQL databases. And the business benefits of big data are potentially revolutionary. However, at its essence, big data requires an architecture that acquires data from multiple data sources, organizes and stores that data in a suitable format for analysis, enables users to efficiently analyze the data and ultimately helps to drive business decisions. These are the exact same principles that IT organizations have been following for data warehouses for years.




The new information architecture that enterprises will pursue in the big data era is an extension of their previous data warehouse architectures. The data warehouse, built upon a relational database, will continue to be the primary analytic database for storing much of a company’s core transactional data, such as financial records, customer data, and sales transactions. The data warehouse will be augmented by a big-data system, which functions as a ‘data lake’. This will be the repository for the new sources of large volumes of data: machine-generated log files, social- media data, and videos and images -- as well as the repository for more granular transactional data or older transactional data which is not stored in the data warehouse. Even though the new information architecture consists of multiple physical data stores (relational, Hadoop, and NoSQL), the logical architecture is a single integrated data platform, spanning the relational data warehouse and the Hadoop-based data lake.

Technologies such as Oracle Big Data SQL make this distributed architecture a reality; Big Data SQL1 provides data virtualization capabilities, so that SQL can be used to access any data, whether in relational databases or Hadoop or NoSQL. This virtualized SQL layer also enables many other languages and environments, built on top of SQL, to seamlessly access data across the entire big data platform.

Oracle Database 12c Release 2 and Oracle Exadata: A Data Warehouse as a Foundation for Big Data

Even as new big data architectures emerge and mature, business users will continue to analyze data by directly leveraging and accessing data warehouses. The rest of this paper describes how Oracle Database 12c Release 2 provides a comprehensive platform for data warehousing that combines industry-leading scalability and performance, deeply-integrated analytics, and advanced workload management – all in a single platform running on an optimized hardware configuration.


Hot cloning and refreshing PDBs in Oracle 12cR2

Exadata

The bedrock of a solid data warehouse solution is a scalable, high-performance hardware infrastructure. One of the long-standing challenges for data warehouses has been to deliver the IO bandwidth necessary for large-scale queries, especially as data volumes and user workloads have continued to increase. While the Oracle Exadata Database Machine is designed to provide the optimal database environment for every enterprise database, the Exadata architecture also provides a uniquely optimized storage solution for data warehousing that delivers order-of- magnitude performance gains for large-scale data warehouse queries and very efficient data storage via compression for large data volumes. A few of the key features of Exadata that are particularly valuable to data warehousing are:

  • » Exadata Smarts Scans. With traditional storage, all database intelligence resides on the database servers. However, Exadata has database intelligence built into the storage servers. This allows database operations, and specifically SQL processing, to leverage the CPU’s in both the storage servers and database servers to vastly improve performance. The key feature is “Smart Scans”, the technology of offloading some of the data-intensive SQL processing into the Exadata Storage Server: specifically, row-filtering (the evaluation of where-clause predicates) and column-filtering (the evaluation of the select-list) are executed on Exadata storage server, and a much smaller set of filtered data is returned to the database servers. “Smart scans” can improve the query performance of large queries by an order of magnitude, and in conjunction with the vastly superior IO bandwidth of Exadata’s architecture delivers industry-leading performance for large-scale queries.
  • » Exadata Storage Indexes. Completely automatic and transparent, Exadata Storage Indexes maintain each column’s minimum and maximum values of tables residing in the storage server. With this information, Exadata can easily filter out unnecessary data to accelerate query performance.
  • » Hybrid Columnar Compression. Data can be compressed within the Exadata Storage Server into a highly efficient columnar format that provides up to a 10 times compression ratio, without any loss of query performance. And, for pure historical data, a new archival level of hybrid columnar compression can be used that provides up to 40 times compression ratios.

Partner Webcast - Oracle Cloud Machine Technical Overview (Part1)



Partner Webcast - Oracle Cloud Machine Technical Overview (Part 2)


Oracle Database In-Memory

While Exadata tackles one major requirement for high-performance data warehousing (high-bandwidth IO), Oracle Database In-Memory tackles another requirement: interactive, real-time queries. Reading data from memory can be orders of magnitude faster than reading from disk, but that is only part of the performance benefits of In-Memory: Oracle additionally increases in-memory query performance through innovative memory-optimized performance techniques such as vector processing and an optimized in-memory aggregation algorithm. Key features include:

  • » In-memory (IM) Column Store. Data is stored in a compressed columnar format when using Oracle Database In-Memory. A columnar format is ideal for analytics, as it allows for faster data retrieval when only a few columns are selected from a table(s). Columnar data is very amenable to efficient compression; in-memory data is typically compressed 2-20x, which enables larger volumes of raw data to be stored in the in-memory column store.
  • » SIMD Vector Processing. When scanning data stored in the IM column store, Database In-Memory uses SIMD vector processing (Single Instruction processing Multiple Data values). Instead of evaluating each entry in the column one at a time, SIMD vector processing allows a set of column values to be evaluated together in a single CPU instruction. In this way, SIMD vector processing enables the Oracle Database In-Memory to scan and filter billion of rows per second.
  • » In-Memory Aggregation. Analytic queries require more than just simple filters and joins. They require complex aggregations and summaries. Oracle Database In-Memory provides an aggregation algorithm specifically optimized for the join-and-aggregate operations found in typical star queries. This algorithm allows dimension tables to be joined to the fact table, and the resulting data set aggregated, all in a single in-memory pass of the fact table.

Oracle Database In-Memory is useful for every data-warehousing environment. Oracle Database In-Memory is entirely transparent to applications and tools, so that it is simple to implement. Unlike a pure in-memory database, not all of the objects in an Oracle database need to be populated in the IM column store. The IM column store should be populated with the most performance-critical data, while less performance-critical data can reside on lower cost flash or disk. Thus, even the largest data warehouse can see considerable performance benefits from In- Memory.

Query Performance

Oracle provides performance optimizations for every type of data warehouse environment. Data warehouse workloads are often complex, with different users running vastly different operations, with similarly different expectations and requirements for query performance. Exadata and In-Memory address many performance challenges, but many other fundamental performance capabilities are necessary for enterprise-wide data warehouse performance.
Oracle meets the demands of data warehouse performance by providing a broad set of optimization techniques for every type of query and workload:

  • » Advanced indexing and aggregation techniques for sub-second response times for reporting and dashboard queries. Oracle’s bitmap and b-tree indexes and materialized views provide the developer and DBA’s with tools to make pre-defined reports and dashboards execute with fast performance and minimal resource requirements.
  • » Star query optimizations for dimensional queries. Most business intelligence tools have been optimized for star- schema data models. The Oracle Database is highly optimized for these environments; Oracle Database In- Memory provides fast star-query performance leverage its in-memory aggregation capabilities. For other database environments, Oracle’s “star transformation” leverages bitmap indexes on the fact table to efficiently join multiple dimension tables in a single processing step. Meanwhile, Oracle OLAP is a complete multidimensional analytic engine embedded in the Oracle Database, storing data within multidimensional cubes inside the database accessible via SQL. The OLAP environment provides very fast access to aggregate data in a dimensional environment, in addition to sophisticated calculation capabilities (the latter is discussed in a subsequent section of this paper).
  • » Scalable parallelized processing. Parallel execution is one of the fundamental database technologies that enable users to query any amount of data volumes. It is the ability to apply multiple CPU and IO resources to the execution of a single database operation. Oracle’s parallel architecture allows any query to be parallelized, and Oracle dynamically chooses the optimal degree of parallelism for every query based on the characteristics of the query, the current workload on the system and the priority of requesting user.
  • » Partition pruning and partition-wise joins. Partition pruning is perhaps one of the simplest query-optimization techniques, but also one of the most beneficial. Partition pruning enables a query to only access the necessary partitions, rather than accessing an entire table – frequently, partition-pruning alone can speed up a query by two orders of magnitude. Partition-wise joins provide similar performance benefits when joining tables that are partitioned by the same key. Together these partitioning optimizations are fundamental for accelerating performance for queries on very large database objects.

Oracle Database 12c Release 2 Rapid Home Provisioning and Maintenance


The query performance techniques described here operate in a concerted fashion, and provide multiplicative performance gains. For example, a single query may be improved by 10x performance via partition-pruning, by 5x via parallelism, by 20x via star query optimization, and by 10x via Exadata smart scans – a net improvement of 10,000x compared to a naïve SQL engine.
Orchestrating the query capabilities of the Oracle database are several foundational technologies. Every query running in a data warehouse benefit from:

  • » A query optimizer that determines the best strategy for executing each query, from among all of the available execution techniques available to Oracle. Oracle’s query optimizer provides advanced query-transformation capabilities, and, in Oracle Database 12c, the query optimizer adds Adaptive Query Optimization, which enables the optimizer to make run-time adjustments to execution plans.
  • » A sophisticated resource manager for ensuring performance even in databases with complex, heterogeneous workloads. The Database Resource Manager allows end-users to be grouped into ‘resource consumer groups’, and for each group, the database administrator can set policies to govern the amount of CPU and IO resources that can be utilized, as well as specify policies for proactive query governing, and for query queuing. With the Database Resource Manager, Oracle provides the capabilities to ensure that data warehouse can address the requirements of multiple concurrent workloads, so that a single data warehouse platform can, for example, simultaneously service hundreds on online business analysts doing ad hoc analysis in a business intelligence tool, thousands of business users viewing a dashboard, and dozens of data scientists doing deep data exploration.
  • » Management Packs to automate the ongoing performance tuning of a data warehouse. Based upon the ongoing performance and query workload, management packs provide recommendations for all aspects of performance, including indexes and partitioning.

More Information:

http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

http://www.oracle.com/webfolder/technetwork/tutorials/obe/db/12c/r1/poster/OUTPUT_poster/poster.html

https://docs.oracle.com/en/database/

https://docs.oracle.com/database/122/ADMIN/title.htm

http://docs.oracle.com/database/121/CNCPT/cdbovrvw.htm#CNCPT89234

https://docs.oracle.com/database/122/whatsnew.htm

https://docs.oracle.com/database/122/NEWFT/title.htm

https://docs.oracle.com/database/122/NEWFT/toc.htm

https://docs.oracle.com/database/122/INMEM/title.htm

http://www.oracle.com/technetwork/database/enterprise-edition/downloads/oracle12c-windows-3633015.html

https://docs.oracle.com/database/122/LADBN/toc.htm#LADBN-GUID-2404CE5F-6894-4B26-9213-8A47DC262109

http://www.oracle.com/us/corporate/analystreports/ovum-cloud-first-strategy-oracle-db-3520721.pdf

The NEW Oracle Database Appliance Portfolio   https://go.oracle.com/LP=55375?elqCampaignId=52477&src1=ad:pas:go:dg:oda&src2=wwmk160603p00096c0015&SC=sckw=WWMK160603P00096C0015&mkwid=sFw6OzrF5%7Cpcrid%7C215765003921%7Cpkw%7Coracle%20database%7Cpmt%7Cp%7Cpdv%7Cc%7Csckw=srch:oracle%20database

https://broadcast.oracle.com/odatouchcastEN

Oracle Database 12c Release 2 - Get Started with Oracle Database   https://docs.oracle.com/database/122/index.htm

http://www.oracle.com/technetwork/database/security/overview/index.html

http://www.oracle.com/technetwork/database/bi-datawarehousing/data-warehousing-wp-12c-1896097.pdf

http://www.oracle.com/technetwork/database/upgrade/overview/upgrading-oracle-database-wp-122-3403093.pdf

http://www.oracle.com/technetwork/database/upgrade/overview/index.html