INSTALLING C*HOW TO PROPERLY DO IT AND KEEP IT ALL ROLLING ALONG (FAST!!!)

10100101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110101010100011

SO, WHO AM I?

Engage, Discover, Monetize.

victor@viafoura.com

@VictorFAnjos

WHO AM I ANYWAY?

Engage, Discover, Monetize.

• Bio-Informatics Data Scientist

• Employee Retentions Analytics

• Data Warehouse Specialist

• System Operations / DevOps

• Founder & Lead Technologist

• Real Estate Investor and Broker

• Presenter, Speaker, Organizer

• Data Scientist & Architect

10100101011110101101010100101010101101101010101000111011010010101111010110101010010101010110110101010100011101101101101010101010010101111010110101010010101010110110101010100011

@

Our Linux Distro of ChoiceTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Ubuntu(this demo uses Precise --- 12.04)

http://www.ubuntu.com/download/server

#TCUG@@

IPTables / PortsTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

#TCUG@

NTPTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Clocks that are in SYNC are deeply important in a cluster.

Make sure to install NTP and configure it to a known server:

# sudo apt-get -y install ntp# sudo /etc/init.d/ntp stop# sudo ntpdate pool.ntp.org# sudo /etc/init.d/ntp start

#TCUG@@

#TCUG@

Linux Volume ManagerTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Split up your disks…

# sudo apt-get -y install lvm2

Find out which disks you have…

# lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTxvda1 202:1 0 8G 0 disk /xvdb 202:16 0 420G 0 diskxvdc 202:32 0 420G 0 disk

#TCUG@@

#TCUG@

Linux Volume Manager contTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Create Physical Volume(s)

# sudo pvcreate /dev/xvdb

Create Volume Group(s)

# sudo vgcreate cassandra /dev/xvdb

Physical volume "/dev/xvdb" successfully created

Volume group "cassandra" successfully created

Create Logical Volume(s)

# sudo lvcreate --size 100G -n data cassandraLogical volume "data" created

# sudo lvcreate --size 10G -n clog cassandraLogical volume ”clog" created

#TCUG@@

#TCUG@

XFSTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

#TCUG@

CREATE / MOUNT FSTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

# sudo apt-get -y install xfsprogs

# sudo mkfs.xfs /dev/cassandra/data# sudo mkfs.xfs /dev/cassandra/clog

# sudo mkdir -p /var/lib/cassandra/data# sudo mkdir -p /var/lib/cassandra/commitlog

# sudo chown -R cassandra:cassandra /var/lib/cassandra/

Ensure your Ubuntu install has XFS

Create the Filesystem on both the Data and Commitlog Volumes

Create the containing directories

Make sure the user Cassandra owns them

#TCUG@@

#TCUG@

CREATE / MOUNT FS contTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

# df -lhFilesystem Size Used Avail Use% Mounted on/dev/xvda1 7.9G 1.3G 6.2G 18% /udev 3.7G 8.0K 3.7G 1% /devtmpfs 1.5G 200K 1.5G 1% /runnone 5.0M 0 5.0M 0% /run/locknone 3.7G 0 3.7G 0% /run/shm/dev/mapper/cassandra-data 504G 33M 504G 1% /var/lib/cassandra/data/dev/mapper/cassandra-clog 168G 33M 168G 1% /var/lib/cassandra/commitlog

# ls -l /var/lib/cassandra/total 4drwxr-xr-x 2 cassandra cassandra 78 Aug 7 13:04 commitlogdrwxr-xr-x 6 cassandra cassandra 73 Aug 7 13:05 datadrwx------ 2 cassandra cassandra 4096 Aug 7 13:04 saved_caches

Your directory listing should end up like this:

#TCUG@@

#TCUG@

DATASTAX INSTALLTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Enable the Datastax Repository

# echo "deb http://debian.datastax.com/community stable main" | sudo tee -a /etc/apt/sources.list

# curl -L http://debian.datastax.com/debian/repo_key | sudo apt-key add -

Install DSC12 (Cassandra 1.2) --- Datastax most recent release

# sudo apt-get –y install dsc12

#TCUG@@

#TCUG@

DATASTAX INSTALL contTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

Many Cassandra Options Available, but for Bare Minimum…

# sudo sed -i -e "/^rpc_address/c\rpc_address: 0.0.0.0" \-e "/^initial_token/c# initial_token:" \/etc/cassandra/cassandra.yaml

Start it all up!!!

# sudo service cassandra start

Check how it’s doing

#TCUG@@

#TCUG@

NICE TO HAVETHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

DATASTAX OPSCENTER-FREE# sudo apt-get install -y libssl0.9.8 opscenter-free# sudo sed -i -e 's/interface = 127.0.0.1/interface = 0.0.0.0/’ \

/etc/opscenter/opscenterd.conf# sudo service opscenterd start

#TCUG@@

#TCUG@

ALL TOGETHER NOWTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

https://gist.github.com/vanjos/5481606

A Gist to do EVERYTHING we did in this Meetup in less than 90 seconds, completely automated!

#TCUG@

HOW ABOUT A FULL RING?THANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

https://github.com/pcmanus/ccm

CCM (for Cassandra Cluster Manager ... or something)

My Wrapper to auto-build it and give it some extra disk space…

https://gist.github.com/vanjos/6169734# df -lhFilesystem Size Used Avail Use% Mounted on/dev/xvda1 7.9G 1.5G 6.1G 19% /udev 3.7G 8.0K 3.7G 1% /devtmpfs 1.5G 208K 1.5G 1% /runnone 5.0M 0 5.0M 0% /run/locknone 3.7G 0 3.7G 0% /run/shm/dev/mapper/cassandra-data1 168G 33M 168G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node1/data/dev/mapper/cassandra-clog1 84G 33M 84G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node1/commitlogs/dev/mapper/cassandra-data2 168G 33M 168G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node2/data/dev/mapper/cassandra-clog2 84G 33M 84G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node2/commitlogs/dev/mapper/cassandra-data3 168G 33M 168G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node3/data/dev/mapper/cassandra-clog3 84G 33M 84G 1% /home/ubuntu/.ccm/My_DEV_Cluster/node3/commitlogs

# ccm node1 statusDatacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Owns Host ID Token RackUN 127.0.0.1 66.95 KB 33.3% 6282dab3-a276-49de-b42a-18f552f51cc9 -9223372036854775808 rack1UN 127.0.0.2 79.02 KB 33.3% 015a5bd4-8b58-4770-9c55-8979faba6ee9 -3074457345618258603 rack1UN 127.0.0.3 60.58 KB 33.3% 35f82028-1aad-4bab-beee-19f0e50c82a1 3074457345618258602 rack1

#TCUG@

SHAMELESS PLUGTHANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

Data for Good, in affiliation with DataKind.org, brings together leading data scientists with high impact social organizations through a comprehensive, collaborative approach that leads to shared insights, greater understanding, and positive action through "data in the service of humanity".

DataKind.org leads a community of pioneering data scientists with the talent, commitment, and energy to open doors & inspire a new way of using the skills and tools of corporations & governments, to meet the needs of the NFP/NGO and social innovation sector.

meetup.com/DataforGood

@Data_For_Good

#TCUG@

QUESTIONS?THANKS

TO

@Datastax

@RyersonDMZ

@Viafoura

@VictorFAnjos

#TCUG@@

https://github.com/vanjos

@VictorFAnjos

victor@viafoura.com