1. W3Schools – SQL Tutorial

W3Schools is one of the largest web developer sites that you can find on the internet. The website provides a multitude of tutorials you can use to develop your skills and SQL is one of them. On this website, you can learn how to use SQL in SQL Server, MySQL, Oracle, and other systems. What’s practical about this site is the quiz feature where you can test your progress, identify strengths and weaknesses, and improve the learning experience.

2. SQLBolt

SQLBolt is, essentially, a series of interactive lessons and exercises that are created to help users learn SQL easily. The lessons and topics found on this site are comprehensive and they cover all the important details of using SQL. This resource is particularly useful for beginners with 19 easy but important lectures that you should know before you move on to more complex details. They also started adding intermediate lessons, at this point only 3 are available but we can expect more soon.

3. Vertabelo Academy

Vertabelo Academy provides interactive SQL courses right in your browser. Each course features extensive practice material that you can use to enhance your skills and build confidence. The website offers three types of courses: free, paid, beta (works in progress that you can use for free to practice). Vertabelo Academy teaches you about SQL queries, table creation, and data management.

Using this website is easy; you start each item with instructions and examples and when you’re ready, it’s time to move on to exercises. Here, you also have the opportunity to discuss the course with other users and trade experiences.

4. Codeacademy

Codeacademy is an online platform that provides various free coding courses in programming languages. The site is dedicated to providing an optimal learning experience and you can use it to learn how to manage data with SQL. Codeacademy’s LearnSQL is free and interactive. The platform covers the basics of database essentials including queries, tables, aggregate functions, developing advanced database queries, among other things.

Each lesson is divided into three panels containing a description of the exercise, an interactive SQL command line, and a visual representation of the database schema with the result of the query. Check your knowledge with a quiz and see how far you’ve come. To take the course, you have to register using your email address and Facebook or Google account.

5. Udemy

Udemy is a great online resource with a mission to “help anyone learn anything”. SQL courses on this site are paid, but frequent promotions bring prices down and you can find an ideal course regardless of your budget. What’s beneficial about courses at Udemy is that you can opt for the one that perfectly matches your current skills and SQL knowledge.

6. Khan Academy “Intro to SQL”

Khan Academy offers personalized learning dashboard, a lot of practice exercises, and micro-lectures in the form of YouTube videos. This allows you to study at your own pace and develop SQL skills gradually. Unlike many other resources, you can adapt this one to your needs and preferences.

The entire course contains 5 parts starting with basics and leading you all the way up to more advanced lessons. You don’t have to register in order to watch videos, but if you have some questions or want to take part in discussions, then you will have to sign in.

7. SQLZoo

SQLZoo is ideal for people who prefer extensive personal support and a more thorough approach to lessons. Of course, lessons are interactive and the site is free to use. Here, SQL course comes with live interpreters and interactive exercises for different types of databases. All tutorials come in step-by-step format and you also have the option to use live chat, test your knowledge with a quiz, and the content is available without registration.

8. Stanford University

Yes, THAT Stanford University provides an online self-paced course with video tutorials that you can use to learn basic SQL skills. To get started, all you have to do is to select the course and there are plenty of options including querying databases, SQL advanced features such as indexes and transactions, constraints and triggers, online analytical processing, recursion in SQL, and many others.

9. SQL Problems and Solutions

This unique platform acts as an interactive textbook which allows you to visualize tables and execute queries using a sample database. This tutorial explains the fundamental concepts and constructs of SQL. At the same time, it displays examples for different levels of expertise in order to help you learn better and move on to other lessons. Once you’ve learned all the lessons, put your skills to a test using a sister site SQL Exercises.

Bottom line

If you have ever wanted to learn SQL but didn’t know how then resources from this post will help you out. They cover the basics, move to more advanced lessons and courses, and allow you to test your skills using quizzes and sample databases. These sites only prove that you don’t need to spend a fortune to learn more about SQL, you can do it for free.

The post 9 useful resources for those who wants to know more about SQL appeared first on Big Data Made Simple - One source. Many perspectives..

For starters, with poorly written queries, you cannot guarantee users lightning-fast retrieval times. Your servers will also be plagued by problems from day one. And in today’s digital world, these are mistakes you cannot afford to make. But, how do you avoid making these mistakes? Here are tips on how to go about it.

1. Failing to Review Your Data Model

Your data model determines how users access data. So, think your model through right from the beginning. If you do not, you will have to deal with unwieldy queries and complicated code down the line, and both impact negatively on performance. An easy way to figure out which queries are needed to access data is to print out your data model.

Or, better still, have a data modelling tool do it for you. A print-out or modelling tool lets you see what you are up against. You are, therefore, in a better position to simplify code, increase coding time, increase accuracy, and improve performance.

2. Failing to Consider Your Technique

What technique do you use? Is it cursor logic, or set-based logic? There is no easy answer to this particular question: it all depends on the performance that best suits your needs. Take set-based logic, for instance. It is the obvious choice for database access. After all, an SQL Server is designed for it. But, cursor logic can in some instances outperform based logic. The key is not to use one technique when the other would be better.

3. Not Using Old Coding Techniques

When you use tried-and-tested coding techniques, you seldom land in trouble. Even coding methods you learned from SQL Server 2005 can prove useful today. Try to use the TRY…CATCH error handling technique in your coding. The results may surprise you. Using Common Table Expressions for hierarchies, or the Common Language Runtime (CLR) database engine may also leave you surprised.

If you need help brushing up on old techniques, do some revision and look for some articles on line. There are plenty out there. Here and here are a couple of SQL examples.

4. Not Taking Advantage of Peer Review

Before deploying your query plans, you should have someone else review it. Chances are that other people will see what you have missed. Their reviews on your indexes and query performance often help you to further improve your code. They could also learn a thing or two from you in the process, and vice-a-versa.

5. Failing to Test Your Queries

Developers hate having to test code. First, it’s rigorous. And second, the testing environment (hardware and data) rarely match the real production environment. But testing is a necessary, and unavoidable, part of coding. So, thoroughly test your code, and where possible, try to mimic the final production environment as closely as possible. Remember, your queries might perform well with a few hundred records, but not against millions in the final environment.

Conclusion

Queries determine the speed and performance of an SQL database. So, try to avoid common mistakes such as not reviewing your data model, or failing to consider which technique to use. Others are failing to use old coding techniques, not taking advantage of peer review mechanisms, and failing to test your queries.

The post 5 common SQL query design mistakes to avoid at all costs appeared first on Big Data Made Simple - One source. Many perspectives..

Nature argues that researchers must ensure information is reliably encoded and that nucleotide strings can be produced economically and efficiently, as well.  We’ll eventually have no choice, though, since—according to BackupAssist—“the digital data we need to store is increasing at an unsustainable rate for our current hardware.”  This seems like all the more reason to streamline data use and make sure no more is stored than is absolutely necessary.  It does seem, though, that we’re heading toward a kind of science fiction version of reality, in a way.

Scientific American quotes Microsoft’s Karin Strauss envisioning what this DNA storage and retrieval might look like via “archival DNA storage services,” within the next decade: “You could open your browser and upload files to their site or get your bytes back, like cloud storage,” she says.  “Or, with as yet unrealized breakthroughs in DNA synthesis and sequencing, ‘you could buy a DNA drive instead of a disk drive.’”

So there’s the future, and there’s what we have now.  There’s network-attached storage and online or cloud-based storage like DropBox, Google Drive, and iCloud.  For more small-scale data storage, there are flash memory drives and external hard drives.  However, businesses considering cloud-based storage are often concerned with data security; CIO Review identifies SkyHigh Networks and Netskope as two cloud-security companies that are filling the gap.  This concern with security will continue to be a prominent factor as cloud-based storage and backup become the norm—as opposed to on-site, enterprise software.

The problem with many of these cloud-based services is also what makes them convenient: that is, because they are accessible from anywhere, they are also more prone to hacking when being accessed from a location other than the workplace: say a public Internet café or a personal home computer utilized by multiple users.  Another possible point of weakness is the prevalence of BYOD in the workplace, as of late.  If an employee signs into network drives from a personal computer or smartphone while at work, then transmits unsecured data using their own devices, that data will be more vulnerable to hackers and other people outside the network.   Password-protected documents and devices will help ensure security of information, regardless of the device being used.

Microsoft provides a guide to different backup techniques, as well as a bit of information as to different types of backup hardware required to perform backups—information useful to the layman, for practical reference.  In addition to full backups, there are differential backups, incremental backups, and daily backups.  Microsoft recommends doing a combination of full and partial backups on a weekly basis.

For now, the most trending form of data storage and backup come in the form of cloud-based services—though the transition is a slow one.  Antony Adshead predicts that “The tipping point at which public cloud operations attain a 50% share of IT workloads will come in 2030.  Until then, and beyond, we face ‘decades of a hybrid [cloud] world.’”  The hybrid cloud combines a few features of private and public clouds, while also allowing for advanced customization; you can pick and choose which bits of information are stored on the private versus the public side, depending on the level of sensitivity.

*   *   *

As we move into the future, cloud-based computing is likely to become more and more prominent, allowing us to access the majority of our information for work and personal use from anywhere.  The need for effective cybersecurity will also increase, but cloud-based data platform companies are doing a good job of anticipating these concerns by acquiring cybersecurity companies—known in the ‘cloud-world’ as cloud access security brokers (CASB)—at an increased rate.  Expect to see more merging of data storage platforms with security-based companies as the move to cloud becomes increasingly widespread.

The post DNA vs modern backup methods: The future of data storage appeared first on Big Data Made Simple - One source. Many perspectives..

To avoid these potential pitfalls, we are going to take a closer look at some of the most widely used storage engines supported by MySQL (as of version 5.7).

Supported Storage Engines

What are my options?

By default, MySQL 5.7 supports ten storage engines (InnoDB, MyISAM, Memory, CSV, Archive, Blackhole, NDB, Merge, Federated, and Example). To see which ones are available and supported by your server, use this command:

mysql> SHOW ENGINES\G

This will output a list of storage engines and tell you which are available, not available, or which is currently set to the default. The “Support:” column will display ‘YES’, ‘NO’, or ‘DEFAULT’, respectively.

In some applications, the need may arise to have different storage engines for different tables in the same database. This is an example of why you need to carefully plan the data model for your application. In most cases, however, only one storage engine will be needed.

Storage Engine Capabilities

What are they good at?

Let’s take a closer look at some of the most commonly used storage engines. This will give us an idea of what each engine was designed to do and how they can best be used to serve our business goals.

InnoDB: The default option in MySQL 5.7, InnoDB is a robust storage engine that offers:

• Full ACID compliance
• Commit, rollback, and crash-recovery
• Row-level locking
• FOREIGN KEY referential-integrity constraints
• Increase multi-user concurrency (via non-locking reads)

With the above functionality that InnoDB offers, it is obvious why it is the default engine in MySQL. It is an engine that performs well and offers many of the required attributes that any database would need. However, a comprehensive discussion of all of its capabilities is outside the scope of this article. This is the engine that will most likely be used in the majority of applications.

MyISAM: The functionality that sets MyISAM apart is its capability for:

• full text search indexes
• table-level locking
• lack of support for transactions

Though it is a fast storage engine, it is best suited for use in read-heavy and mostly read applications such as data warehousing and web applications that don’t need transaction support or ACID compliance.

NDB (or NDBCLUSTER): If a clustered environment is where your database will be working, NDB is the storage engine of choice. It is best when you need:

• Distributed computing
• High-redundancy
• High-availability
• The highest possible uptimes

Take note that support for NDB is not included in the distribution of standard MySQL Server 5.7 binaries. You will have to update to latest binary release of MySQL Cluster. Though, if you’re developing in a cluster environment, you probably have the necessary experience to deal with these tasks.

CSV: A useful storage engine when data needs to be shared with other applications that use CSV formatted data. The tables are stored as comma separated value text files. Though this makes sharing the data with scripts and applications easier, one drawback is that the CSV files are not indexed. So, the data should be stored in an InnoDB table until the Import/Export stage of the process.

Blackhole: This engine accepts but does not store data. Similar to the UNIX /dev/null, queries always return an empty set. This can be useful in a distributed database environment where you do not want to store data locally or in performance or other testing situations.

Archive: Just as the name implies, this engine is excellent for seldom-referenced historical data. The tables are not indexed and compression happens upon insert. Transactions are not supported. Use this storage engine for archiving and retrieving past data.

Federated: This storage engine is for creating a single, local, logical database by linking several different physical MySQL servers. No data is stored on the local server and queries are automatically executed on the respective remote server. It is perfect for distributed data mart environments and can vastly improve performance when using MySQL for analytical reporting.

Designating a storage engine

How do I change which storage engine is used?

The storage engine that is used is established upon table creation. As previously stated, InnoDB is the default storage engine in MySQL versions 5.5 and higher. If you would like to use a different one, it is best to do this within your CREATE TABLE statement. For instance, let’s say that you have identified a table that needs use the CSV storage engine. Your overly simplified CREATE TABLE statement might look like this:

mysql> CREATE TABLE Shared_Data (

    -> Data_ID INTEGER NOT NULL,

    -> Name VARCHAR(50) NOT NULL,

    -> Description VARCHAR(150)

    -> ) ENGINE=’CSV’;

After which we would perform an INSERT statement as usual:

mysql> INSERT INTO Shared_Data VALUES

-> (1,’device one’, ‘the latest version of the best tech’),

-> (2,’device two’, ‘the fastest one on the market’);

Upon success, if you inspect the database directory, there should now be a ‘Shared_Data.CSV’ file in it that contains the records you have inserted into the Shared_Data table.

The same methodology can be used for any one of the many storage engines that MySQL supports. Though it is possible to change the storage engine after a table has been created with an ALTER TABLE statement, it is best practice to plan accordingly and set it in the beginning.

In Closing

MySQL has many options

As you can see, MySQL offers support for storage engines designed to handle very different tasks in many different environments. Identifying which engines to use and when to use them can help us avoid unnecessary complications and performance issues as our applications scale.

Whether you need 99.999% uptime and reliability on your distributed computing cluster or you need ACID compliant transaction support with FOREIGN KEY constraints, MySQL has a storage engine to suit your needs.

As always, proper planning and identification of your project goals and requirements is the best way to accurately identify which storage engines are best suited for your application. Hopefully, this article serves as a useful starting point for helping you in that respect.

Originally appeared on Sisense blog.

The post Database corner: Beginner’s guide to Mysql storage engines appeared first on Big Data Made Simple - One source. Many perspectives..

Relational Database

In the first part of this blog series, I talked about ACID properties. These properties are important to maintain a tight transactional integration. There are some industries like banking, retail, etc., where each transaction requires ACID properties. In bank transactions, if one account is credited, another should be debited. The partial update is never allowed, as it will affect the data integrity — Oracle, SQL Server, MySQL and other RDBMS are used in this scenario.

Non-Relational Database (NoSQL DB)

In the first part of this blog series, I also talked about BASE properties. These are important to keep data consistent across all nodes in a database. Any information that does not require strict data integrity can be stored in an NoSQL DB. For instance, contents of a Search Engine System can be stored in a non-relational database, because it is easy to retrieve information quickly. A good example of a search engine system is Google. Google usually stores its cached web pages in a web layer that is refreshed periodically. These databases can store terabytes of historical data (say credit card transactions of a bank, for the past 5 years) in a distributed environment. It is easy to do analyse and mine data in an NoSQL DB using SQL-Like HIVE data warehouse software. NoSQL DBs can be used to store massive volumes of unstructured data and are suitable for text analysis too.

I Have Listed Some Top Organisations Who Use These Databases:

Relational Databases

SQL Server: LG Electronics, MySpace, Hilton Hotels.

ORACLE: British Telecom, MasterCard, Reliance Ltd.

MySQL: Facebook, Twitter, LinkedIn. Facebook uses MySQL to store user interaction like status updates, shares, likes, etc.

Non-Relational Databases

CouchBase: LinkedIn, AdAction.

Cassandra: Facebook, Twitter, Digg.

MongoDB: LinkedIn, Pearson.

Neo4j: Cisco, eBay, etc.

As you have seen, companies like Facebook, Twitter and LinkedIn use both Relational and Non-Relational Databases, based on their requirements.

Now let me return to the first part of this series, and answer the following questions:

Are relational databases capable of handling big data?

Are relational databases scalable?

Are relational databases suited for the modern age data requirements? Such as Real-time analytics, dealing with unstructured data?

The answer to all these questions is a vehement “YES”. Relational databases are not going away in this social world. Based on the nature and complexity of the data set, the right database should be used. Both Relational and Non-Relational Databases have their own advantages and disadvantages. The correct environment set up can make use of relational and non-relational data bases in a proper way like how Facebook, Twitter and LinkedIn have done it.

The post Relational vs Non-Relational data bases – Part 3 appeared first on Big Data Made Simple - One source. Many perspectives..

1. MySQL supports up to 64 indexes per table. Each index may consist of 1 to 16 columns. The maximum index size is 1000 bytes (767 for InnoDB).

2. The maximum size of a row in a MySQL table is 65,535 bytes. And the maximum value of Signed Integer is 2,147,483,647 and the value of Unsigned Integer is 4,294,967,295. In a mixed table with both CHAR and VARCHAR, MySQL will change the CHAR’s to VARCHAR’s.

3. If a PRIMARY KEY or UNIQUE index consists of only one column that has an integer type, you can also refer to the column as “_rowid” in SELECT statements.

4. To change the value of the AUTO_INCREMENT, use “ALTER TABLE <Tablename> AUTO_INCREMENT = value;” or “SET INSERT_ID = value;”

5. To restrict MySQL from being accessed publicly, use “skip-networking” option in the config file. When it is enabled, MySQL only listens to local socket connections and ignores all TCP ports. And “bind-address” parameter which is set to “127.0.0.1″ restricts the MySQL to be accessible only by the localhost.

6. If the MySQL has many connects established (i.e. a website without persistent connections), you can improve the performance by setting thread_cache_size to a non-zero value. 16 is a good value to start with. Increase the value until your threads_created do not grow very quickly.

7. NO_AUTO_VALUE_ON_ZERO suppresses auto increment for 0. Only NULL generates the next sequence number. This mode can be useful if 0 has been stored in a table’s AUTO_INCREMENT column. (Storing 0 is not a recommended practice, by the way.)

8. The configuration options “innodb_analyze_is_persistent”, “innodb_stats_persistent_sample_pages” and “innodb_stats_transient_sample_pages” provide improved accuracy of InnoDB index statistics, and consistency across MySQL restarts. InnoDB precomputes statistics that help the optimizer decide which indexes to use in a query, by sampling a portion of the index. You can adjust the amount of sampling that InnoDB does for each index. The resulting statistics can now persist across server restarts, rather than being recomputed (and possibly changing) due to restarts and some runtime events. The more accurate statistics can improve query performance, and the persistence aspect can keep query performance stable. When the persistent stats feature is enabled, the statistics are only recomputed when you explicitly run ANALYZE TABLE for the table.

9. InnoDB frees up the memory associated with an opened table to ease the memory load on systems with huge numbers of tables. An LRU algorithm selects tables that have gone the longest without being accessed. To reserve more memory for open tables, increase the value of the –table_definition_cache=# configuration option.

10. Set table_cache parameter to match the number of open tables and concurrent connections. Watch the open_tables value and if it is growing quickly you need to increase the size of “table_cache”. And for the parameter “open_file_limit” set this limit as 20+max_connections+table_cache*2. If you have complex queries “sort_buffer_size” and “tmp_table_size” are likely to be very important. Values will depend on the query complexity and available resources, but 4Mb and 32Mb, respectively are recommended starting points.

Note: These are “per connection” values. So, consider your load and available resource when setting these parameters. For example sort_buffer_size is allocated only if MySQL needs to do a sort, be careful not to run out of memory.

The post 10 interesting facts and tips about MySQL appeared first on Big Data Made Simple - One source. Many perspectives..

1. Always use proper datatype

Use datatypes based on the nature of data. If you use irrelevant datatypes it may consume more space or may lead to errors.

Example: Using varchar (20) to store date time values instead of DATETIME datatype will lead to errors during date time related calculations and there is also possible case of storing invalid data.

2. Use CHAR (1) over VARCHAR(1)

If you are string a single character, use CHAR(1) instead of VARCHAR(1) because VARCHAR(1) will take extra byte to store information

3. Use CHAR datatype to store only fixed length data

Example: Using char(1000) instead of varchar(1000) will consume more space if the length of data is less than 1000

4. Avoid using regional date formats

When you use DATETIME or DATE datatype always use YYYY-MM-DD date format or ISO date format that suits your SQL Engine. Other regional formats like DD-MM-YYY, MM-DD-YYYY will not be stored properly.

5. Index key columns

Make sure to index the columns which are used in JOIN clauses so that the query returns the result fast.

If you use UPDATE statement that involves more than one table make sure that all the columns which are used to join the tables are indexed

6. Do not use functions over indexed columns

Using functions over indexed columns defeats the purpose of index. Suppose you want to get data where first two character of customer code is AK, do not write

SELECT columns FROM table WHERE left (customer_code,2)=’AK’

but rewrite it using

SELECT columns FROM table WHERE customer_code like ‘AK%’

which will make use of index which results to faster response time.

7. Use SELECT * only if needed

Do not just blindly use SELECT * in the code. If there are many columns in the table, all will get returned which will slow down the response time particularly if you send the result to a front end application.

Explicitly type out the column names which are actually needed.

8. Use ORDER BY Clause only if needed

If you want to show the result in front end application, let it ORDER the result set. Doing this in SQL may slow down the response time in the multi user environment.

9. Choose proper Database Engine

If you develop an application that reads data more often than writing (ex: search engine), choose MyISAM storage engine.

If you develop an application that writes data more often than reading (ex: real time bank transactions), choose INNODB storage engine.

Choosing wrong storage engine will affect the performance

10. Use EXISTS clause wherever needed

If you want to check the existence of data, do not use

If (SELECT count(*) from Table WHERE col=’some value’)>0

instead, use EXISTS clause

If EXISTS(SELECT * from Table WHERE col=’some value’)

which is faster in response time.

The post Top 10 best practices in MySQL appeared first on Big Data Made Simple - One source. Many perspectives..

1. Analytics, Data Mining, Predictive Modeling, Artificial Intelligence

2. APACHE HADOOP

3. Apache Hadoop Ecosystem

4. Big Data

5. Big Data Analytics using R

6. Big Data Analytics with R and Hadoop

7. Big data hadoop NOSQL Hive Hbase

8. Big Data Learnings

9. Big Data Malaysia

10. Big Data, Data Science, Data Mining & Statistics

11. BigData/Hadoop Expert

12. Chennai Hadoop and Big Data User Group

13. Data Mining / Machine Learning / AI

14. Data Mining/Big Data

15. Hadoop Administrators

16. Hadoop Developers India

17. Hadoop in Action

18. Hadoop Jobs

19. Hadoop Material

20. Hadoop User Group

21. ORACLE SQL PlSQL Discussions

22. SQL Server Network

23. SQL Server Pune User Group

24. SQLBangalore

25. SQLServerGeeks

The post Top Facebook groups for Analytics, Big Data, Data Mining, Hadoop, NoSQL, Data Science appeared first on Big Data Made Simple - One source. Many perspectives..

For working only with structured data, the comparison is more nuanced. In principle, SQL and Hadoop can be complementary, as SQL is a query language which can be implemented on top of Hadoop as the execution engine.

But in practice, SQL databases tend to refer to a whole set of legacy technologies, with several dominant vendors, optimized for a historical set of applications. Many of these existing commercial databases are a mismatch to the requirements that Hadoop targets.

With that in mind, let’s make a more detailed comparison of Hadoop with typical SQL databases on specific dimensions.

Scale-out instead of scale-up

Scaling commercial relational databases is expensive. Their design is more friendly to scaling up. To run a bigger database you need to buy a bigger machine. In fact, it’s not unusual to see server vendors market their expensive high-end machines as “database-class servers.”

The post Comparing SQL databases and Hadoop appeared first on Big Data Made Simple - One source. Many perspectives..

The straightforward answer is $8k * 12 = $96k. But how confident should you be? Will your conversion rate always be so close to 5%? You could pad the estimate ±20% for safety, guessing at $77k to $115k. If $77k would cover all your expenses, should you feel secure?

This is a question of binomial probability. Using our favorite binomial confidence interval calculator, the 95% confidence interval for your conversion rate is about 2.5% to 9%.

With a confidence interval that wide, you should expect to make somewhere between $48k and $172k. Yikes! You could end up with half of your simple guess, and that’s if your business doesn’t change.

These confidence intervals are very informative, but turning to a calculator for every metric is tedious. If you’ve got hundreds of metrics across dozens of dashboards, it’s downright unsustainable.

Fortunately, the math for calculating confidence interval is simple to implement:

The post How to Calculate Confidence Intervals in SQL appeared first on Big Data Made Simple - One source. Many perspectives..

Scalability

It is an ability of a system that can easily accomadate the rapid incoming data without much performance problems. This is a main factor for any system to provide good scalability. There are two types of scaling methods known as Vertical and Horozontal scaling.

Vertical Scaling

All the Relational database tools support vertical scaling. This is the method of increasing the power of the system by adding additional CPU, memory and disk spaces. So to allow rapid incoming data, the single production server is optimsed to scale up. In this scaling technique there is always a single production server which can be connected by all the applications and users. A cluster environment can be created with some nodes and replicate the data across nodes. Because of ACID properties, all nodes should have the same set of data and data synchronization becomes complicated if there are serveral nodes in the clsuter. This is very optimised for Read scaling. Vertical scaling is also known as scale-up

The benefit of this scaling methodlogy is the tight integration of data and its consistency across the nodes in a cluster. All nodes will have the same set of data and If there is a problem with the production server, another node will automatically be connected by the applications. So this cluster is known as Fail-over cluster.

Horizontal scaling

All the Non-relational database tools support horizontal scaling. This is the method of adding more computers to the network to allow rapid incoming data. It is easy to add more nodes into the cluster to allow data growth. Data are split automatically and processed across nodes in a cluster. This is a distributed data environment. Hadoop Distributed File System (HDFS) is a classical example for this. Horizontal scaling is also known as Scale-out.

The benefit of this scaling technique is that since data are split and replicated across nodes, if any of the nodes goes offline, the application can still have the data from other nodes and this gurantees the availabilty of data at all the time. This method is very useful for the cases where no JOINs are required among the data of the nodes. This is also helpful in seperating data and having them in different geographical locations.

While both these scaling techniques have advantages and disadvantages, a good environment can mix both of these to have outstanding Scale-up and Scal-out. We can have a scale-up read and write database in a single server which requires ACID properties and have a scale-out distributed historical data across several nodes for data mining purpose.

The post Relational Vs Non-Relational databases – Part 2 appeared first on Big Data Made Simple - One source. Many perspectives..

SQL Server 2014 is the latest version of the industry’s most deployed database. After announcing the new software, Nadella shared the company’s path to deliver a platform for the next era of “ambient intelligence.” Ambient intelligence deals with electronic environments that are both sensitive and responsive to people’s presence. It’s one vision for the future of consumer electronics.

“Developing the ability to convert data into the fuel for ambient intelligence is an ambitious challenge. It requires technology to understand context, derive intent and separate signal from noise,” said Nadella. “Building out a comprehensive platform that can enable this kind of ambient intelligence is a whole company initiative that we are uniquely qualified to undertake.”

The Benefits of Big Data

At the launch event, Nadella stressed the importance of a data culture — one that encourages curiosity, action and experimentation — for everyone and every organization. Also showcased were the results of a new IDC study that demonstrates the clear benefits for companies that take a comprehensive data approach. Specifically, these companies realize an additional 60 percent return on data assets. IDC figures it’s a $1.6 trillion opportunity worldwide.

The post Can Microsoft’s SQL Server 2014 Compete with SAP HANA? appeared first on Big Data Made Simple - One source. Many perspectives..