ScaleDB ONE: Let’s Get Started

ScaleDB 15.10 is out. Some users have downloaded it and tested it and we have received pretty positive feedback, but also some requests to have more info and help on how to start. I will try to condense here the basic steps to install and test ScaleDB ONE for the first time.

First of all, some terminology. We have two versions: ScaleDB ONE and ScaleDB Cluster. ScaleDB ONE is meant to be used on a single machine (ONE = One Node Edition), whether it is a VM, a cloud instance or a physical server, whilst ScaleDB Cluster is the full size, multi-node cluster that everybody expects to run for mission critical applications. This means that the typical use cases for ScaleDB ONE are testing and development, data marts and streaming data collection and analysis that are limited by a single server (although you can always replicate your data to another server using the standard MySQL Replication). ScaleDB Cluster instead, is highly available out of the box, with no single point of failure, and scaleable on demand (i.e. you do not need replication to set up availability and a more scalable environment).

From now on, in this post I will refer to ScaleDB as ScaleDB ONE.

Now, some prerequisites. ScaleDB has been tested on CentOS 6.7, CentOS 7.1 and on Ubuntu 14.04.3 Trusty Tahr. 1GB of memory and few GB of free disk space would be enough to test the product but, as for any other databases, the more cores, memory and storage you can add, the better. One interesting aspect is that, if you are planning to store a large amount of data, you will be very pleased with the performance you can get from ScaleDB using magnetic HDDs instead of SSD (but this is a topic for another post).

On CentOS 6.7, you must add nmap and nc, since you will need them later to interact with the ScaleDB daemons. A yum install nmap-ncat should do the trick. If you have done a minimal install of CentOS 7.1, I would also recommend to install nettools.

Another mandatory requirement is the installation of the AIO libraries (with yum install libaio). For Ubuntu 14.04.3, you will be required to install the AIO libraries, with sudo apt-get install libaio1.

The last two steps are not mandatory, but they will make your life easier: create a scaledb user that can be a sudoer and disable the firewall on your testing machine. From now on, I assume that you will log in as scaledb.

Downloading the software

If you have not downloaded ScaleDB ONE yet, you will have to hit few pages on the ScaleDB website, but the process is very straightforward. Just go to scaledb.com, click the Download button, scroll down to the bottom and click another Download button (this time it is grey). The next screen is used to select the type of download. You have two choices:

  • VirtualBox Image, which will allow you to download a OVA (a compressed VirtualBox image file), so everything is self contained in a CentOS 7 image and you do not need to install any software.
  • Tarball, which allows you to decompress and unarchive the ScaleDB product. We do not have Linux packages yet, they will be available soon.

Once you select your favourite download, all you have to do is to fill five fields, then you will receive am email with your personalised link to the downloads. You can keep this link, we will update your environment with new versions. Soon we will also add a yum and a apt repository.

You can use the unique URL to download ScaleDB ONE. You must add /scaledb-15.10/latest-release to the URL or you can simply browse the repository to search the release you need. In the latest-release folder, you will find 3 tarballs:

  1. The ScaleDB ONE UDE: UDE stands for Universal Data Engine. This is the ScaleDB engine. Right now (2015-11-17), the latest UDE tarball is scaledb-15.10.1-13199-ude.tgz.
  2. MariaDB: We use MariaDB as database server. ScaleDB can work on its own, i.e. you can access the engine by using the ScaleDB API, but for the MySQL and MariaDB users we have created a storage engine layer, so ScaleDB is fully accessible from MariaDB. The version available with 15.10 is MariaDB 10.0.14, soon we will release a version that works with the latest MariaDB 10.1. For Ubuntu you can use scaledb-15.10.1-mariadb-10.0.14-glibc2.14.tgz, for CentOS scaledb-15.10.1-mariadb-10.0.14.tgz.

You can donwload the tarballs on your own machine and then copy them to the testing machine or you can download them directly into the testing machine with a wget command.

Assuming that the tarballs are in the root directory of the scaledb user, you can now uncompress and copy the files on /usr/local with these commands:

sudo tar xzvf ~/scaledb-15.10.1-13199-ude.tgz -C /

…and…

sudo tar xzvf ~/scaledb-15.10.1-mariadb-10.0.14.tgz -C /

That’s it! ScaleDB is ready to be used.

In order to make things easier for anybody who wants to test ScaleDB ONE, we have assumed that the software will be installed in /usr/local and we have a predefined configuration. More specifically:

For MariaDB:

  • The base directory is /usr/local/mysql
  • The data directory is /usr/local/mysql/data
  • The configuration file is /usr/local/mysql/my.cnf
  • The admin user is root (no password)

For the ScaleDB engine:

  • The base directory is /usr/local/scaledb
  • The data directory is /usr/local/scaledb/data
  • There are three configuration files:
    • Storage Engine: /usr/local/mysql/scaledb.cnf
    • Storage Node: /usr/local/scaledb/cas.cnf
    • Lock Manager: /usr/local/scaledb/slm.cnf

At this point you can launch ScaleDB ONE with this script:

/usr/local/scaledb/scripts/scaledb_one start

If you add /usr/local/scaledb/scripts to your PATH you would see something like:

scaledb@ONE:~$ scaledb_one start
 Starting ScaleDB CAS server...
 ScaleDB CAS Server started.
 Starting ScaleDB SLM server...
 ScaleDB SLM server started.
 Starting MariaDB Server...
 MariaDB Server started.
 ScaleDB ONE started.
scaledb@ONE:~$

The script starts both MariaDB server and the ScaleDB engine. The ScaleDB engine is formed by two daemons, the CAS Server (Cache Accelerated Storage Server) and the SLM Server (ScaleDB Lock Manager Server). The same script must be used to stop the environment, by simply using scaledb_one stop.

Testing ScaleDB ONE

I will give you more information on how to properly test ScaleDB very soon. In the meantime, let’s just see if it works as expected.

The MariaDB interface has no difference, the only news is the ScaleDB storage engine:

SHOW_ENGINES

Now it is time to test the engine. We can start by creating a table. The command to create a streaming table is a bit different from a standard InnoDB table. Here is an example:

MariaDB [(none)]> CREATE TABLE test.test (
 -> id bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
 -> create_time timestamp NOT NULL,
 -> account char(8) NOT NULL,
 -> store int(10) UNSIGNED NOT NULL,
 -> amount decimal(8,2) NOT NULL,
 -> PRIMARY KEY (id) STREAMING_KEY=YES,
 -> KEY create_time (create_time) RANGE_KEY=SYSTEM )
 -> ENGINE=ScaleDB TABLE_TYPE=STREAMING;
 Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]>

And here is the explanation line by line:

  • Streaming tables are special tables in ScaleDB that are used to store streaming data. They are really fast and work as a constant stream, i.e. you can INSERT the data, you can SELECT to run a query and you can DELETE the oldest data, but you cannot UPDATE any row or DELETE a row with a general condition.
  • The id column is the primary key: streaming tables always require a primary key.
  • The create_time is a timestamp associated to a range key, a special time-based key used in ScaleDB to query by a time interval.
  • The table attributes to add are the engine (ScaleDB) and the table type (STREAMING). At the moment we do not recommend to use any other type of table.

When you have received an OK, you have created your first ScaleDB table – congratulations!

And finally a simple INSERT and SELECT:

MariaDB [(none)]> INSERT INTO test.test VALUES (NULL, NULL, 'A', 1, 100);
Query OK, 1 row affected (1.54 sec)

MariaDB [(none)]> SELECT * FROM test.test;
 +-----+---------------------+---------+-------+--------+
 | id  | create_time         | account | store | amount |
 +-----+---------------------+---------+-------+--------+
 | 256 | 2015-11-16 06:30:24 | A       |     1 | 100.00 |
 +-----+---------------------+---------+-------+--------+
 1 row in set (0.00 sec)

MariaDB [(none)]>

One warning, if you run a SELECT query immediately after the INSERT and you do not see the that you have just inserted, it is a “normal behaviour”. By default, ScaleDB has a time window of 30 seconds that is used to flush a large number of rows in one go. This behaviour can be changed to a more usual OLTP behaviour, but the price is a lower load rate. In ScaleDB Cluster, the rows are safely stored on two servers and they are not lost in case of fault.