Steps to import column family from one cassandra cluster to another using sstableloader
a) Static Column Family Import
1. Export existing column family into a csv file from source cluster.
Login to cqlsh and run following command to export columnfamily into csv
copy columnfamily_name into ‘/home/user/cf.csv’
Also, you can export some specific columnns to csv as follows:
copy columnfamily_name(column1,column2) into ‘/home/user/cf.csv’
2. Generate sstables from csv file as follows:
i) Creating sstable Writer
SSTableSimpleUnsortedWriter columnfamilyWriter = new SSTableSimpleUnsortedWriter(
directory, new RandomPartitioner(), keySpace, columnFamily,
AsciiType.instance, null, 64);
directory, new RandomPartitioner(), keySpace, columnFamily,
AsciiType.instance, null, 64);
• The directory is the directory where to put the sstables ,this directory is object of File class in java
• SSTableSimpleUnsortedWriter buffers whatever input it gets in memory
and “flush” everything in one sstable once the buffer size is reached
ii) Parsing each CSV line as per column types
boolean parse(String line) {
String[] columns = line.split(",");
try {
key= columns[0].trim();
id = Integer.parseInt(columns[1].trim());
name= columns[2].trim();
salary = Long.valueOf(columns[3].trim());
return true; } catch (Exception e) { return false;
}
return true; } catch (Exception e) { return false;
}
• For example, column type of salary is “Long “ , so parsed it
with Long value. Similarly , column type of id is int so parsed it
with Integer value
iii) Generating sstables and closing writer
while ((line = reader.readLine()) != null) {
if (entry.parse(line)) {
columnfamilyWriter.addColumn(bytes("nasport"), bytes(entry.nasport),tmestamp);
columnfamilyWriter.addColumn(bytes("acctstarttime"),bytes(entry.acctstarttime), timestamp);
columnfamilyWriter.addColumn(bytes("bin"), bytes(entry.bin), timestamp);
columnfamilyWriter.addColumn(bytes("bout"), bytes(entry.bout),timestamp);
if (entry.parse(line)) {
columnfamilyWriter.addColumn(bytes("nasport"), bytes(entry.nasport),tmestamp);
columnfamilyWriter.addColumn(bytes("acctstarttime"),bytes(entry.acctstarttime), timestamp);
columnfamilyWriter.addColumn(bytes("bin"), bytes(entry.bin), timestamp);
columnfamilyWriter.addColumn(bytes("bout"), bytes(entry.bout),timestamp);
columnfamilyWriter.addColumn(bytes("src"), bytes(entry.src), timestamp);
}
} columnfamilyWriter.close();
} columnfamilyWriter.close();
3. Use sstableloader to upload the sstables into the column family of the target cluster
$CASSANDRA_HOME/bin/sstableloader –d $target_cassandra_ip /path/to/sstables
Note : sstables have to be in a directory whose name is the name of the
column family of the sstables which should be the sub-directory of a
directory whose name is the name of the keyspace of the column family
For example, /home/user/keyspace_name/column_family_name
b) Dynamic Column Family Import
Now, in case your table is Dynamic , then its csv would be difficult to parse.Also, no need to worry about indexes here. So those can be imported without using sstableloader as follows:
1. Copy the sstables from Cassandra data directory from source to target cluster
For example, cassandra's data directory is /var/lib/cassandra and
column family is testtable in testkeyspace. Then copy all the sstable
from /var/lib/cassandra/testkeyspace/testtable to targetcluster
2. Use sstableloader to upload the sstables into the column family of the target cluster
$CASSANDRA_HOME/bin/sstableloader –d $target_cassandra_ip /path/to/sstables
Note
: sstables have to be in a directory whose name is the name of the
column family of the sstables which should be the sub-directory of a
directory whose name is the name of the keyspace of the column family
For example, /home/user/keyspace_name/column_family_name
