云计算于大数据入门实验三——熟悉常用的 HBase 操作-Toy模板网

这篇具有很好参考价值的文章主要介绍了云计算于大数据入门实验三——熟悉常用的 HBase 操作。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

云计算于大数据入门实验三——熟悉常用的 HBase 操作

实验目的

理解HBase在Hadoop体系结构中的角色

熟练使用HBase操作常用的shell命令

熟悉HBase操作常用的Java API

实验要求

保存程序，并自行存档
最终的程序都必须经过测试，验证是正确的
按照实验报告格式，认真记录实验过程及结果，回答实验报告中的问题。实验报告模板在学习通的资料里面下载。学生提交的实验报告需转换成PDF文件提交

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

实验步骤

Hbase 常用命令

在 Hbase 中建表

create 'student','Sname','Ssex','Sage','Sdept','course'

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

查看表结构

describe 'student'

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

添加数据

put 'student', '95001','Sname','LiYing'
put 'student','95001','course:math','80'

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

查看数据

get 'student','95001'

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

删除数据

delete 'student','95001','Sname:firstName'
deleteall 'student','95001'

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

删除表

disable 'student'      #让表不可用
drop 'student'         #删除表

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

查询历史数据

create 'teacher',{NAME=>'username',VERSIONS=>5}
put 'teacher','91001','username','Mary'
put 'teacher','91001','username','Mary1'
put 'teacher','91001','username','Mary2'
put 'teacher','91001','username','Mary3'
put 'teacher','91001','username','Mary4'
put 'teacher','91001','username','Mary5'
get 'teacher','91001',{COLUMN=>'username',VERSIONS=>3}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

退出 HBase

exit

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

编程实践

例子：创建表，插入数据，查看表中数据

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.util.Bytes;
 
import java.io.IOException;
public class ExampleForHBase {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    public static void main(String[] args)throws IOException{
        init();　　　　//主要操作就是为了连接到数据库hbase
        createTable("student",new String[]{"score"});　　　　//创建表，shell命令：create '表名'，'列族名1'，'列族名2'，'列族名3' ...
        insertData("student","zhangsan","score","English","69"); //shell命令： put 'student'，'张三'，'score：English'，'69'
        insertData("student","zhangsan","score","Math","86");
        insertData("student","zhangsan","score","Computer","77");
        getData("student", "zhangsan", "score","English");
        close();
    }
 
    public static void init(){
        configuration  = HBaseConfiguration.create();
        configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
        try{
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        }catch (IOException e){
            e.printStackTrace();
        }
    }
 
    public static void close(){
        try{
            if(admin != null){
                admin.close();
            }
            if(null != connection){
                connection.close();
            }
        }catch (IOException e){
            e.printStackTrace();
        }
    }
 
    public static void createTable(String myTableName,String[] colFamily) throws IOException {
        TableName tableName = TableName.valueOf(myTableName);
        if(admin.tableExists(tableName)){
            System.out.println("talbe is exists!");
        }else {
            TableDescriptorBuilder tableDescriptor = TableDescriptorBuilder.newBuilder(tableName);
            for(String str:colFamily){
                ColumnFamilyDescriptor family = 
ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes(str)).build();
                tableDescriptor.setColumnFamily(family);
            }
            admin.createTable(tableDescriptor.build());
        } 
    }
 
    public static void insertData(String tableName,String rowKey,String colFamily,String col,String val) throws IOException { 
        Table table = connection.getTable(TableName.valueOf(tableName));
        Put put = new Put(rowKey.getBytes());
        put.addColumn(colFamily.getBytes(),col.getBytes(), val.getBytes());
        table.put(put);
        table.close(); 
    }
 
    public static void getData(String tableName,String rowKey,String colFamily, String col)throws  IOException{ 
        Table table = connection.getTable(TableName.valueOf(tableName));
        Get get = new Get(rowKey.getBytes());
        get.addColumn(colFamily.getBytes(),col.getBytes());
        Result result = table.get(get);
        System.out.println(new String(result.getValue(colFamily.getBytes(),col==null?null:col.getBytes())));
        table.close(); 
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

实验三熟悉常用的HBase 操作

编程实现以下指定功能，并用 Hadoop 提供的 HBase Shell 命令完成相同任务：

列出 HBase 所有的表的相关信息，例如表名；
在终端打印出指定的表的所有记录数据；
向已经创建好的表添加和删除指定的列族或列；
清空指定的表的所有记录数据；
统计表的行数。

列出 HBase 所有的表的相关信息，例如表名：

/**
* 同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
*  重点是：HTableDescriptor hTableDescriptors[] = admin.listTables(); 获取到 表格列表，然后遍历
*/
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class Test_1 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    /**
     * 建立连接
     */
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    /**
     * 关闭连接
     */
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
 
    }
    /**
     * 
     * 查看已有表,通过方法listTables()
     * 
     * @throws IOException
     * 
     */
    public static void listTables() throws IOException {
        init();
        HTableDescriptor hTableDescriptors[] = admin.listTables();
        for (HTableDescriptor hTableDescriptor : hTableDescriptors) {
            System.out.println(hTableDescriptor.getNameAsString());
        }
        close();
    }
    
    public static void main(String[] args) {
        Test_1 t = new Test_1();
        try {
            System.out.println("以下为Hbase 数据库中所存的表信息");
            t.listTables();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

在终端打印出指定的表的所有记录数据；

/**
* 同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
* 重点是： 
* Table table = connection.getTable(TableName.valueOf(tableName));获取到表格对象
* Scan scan = new Scan(); ResultScanner scanner = table.getScanner(scan); 然后通过Scanner对象，获取到ResultScanner扫描结果对象，遍历输出
*/
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
import java.util.Scanner;
public class Test_2 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
 
        }
    }
    /**
     * 
     * 根据表名查找表信息
     * 
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner)
        {
            showCell((result));
        }
        close();
    }
 
    /**
     * 
     * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
    public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
        Test_2 t = new Test_2();
        System.out.println("请输入要查看的表名");
        Scanner scan = new Scanner(System.in);
        String tableName = scan.nextLine();
        System.out.println("信息如下：");
        t.getData(tableName);
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

向已经创建好的表添加和删除指定的列族或列:

put 'student','95003','Sname','wangjinxuan' 　　　　(添加列)
 
put 'student','95003','Sname:nickName','wang'　　　　(添加列族)
 
put 'student','95003','Sname:firstName','jinxuan'    (添加列族)
 
put的反向操作的delete：
 
delete 'student' ,’95003’,’Sname’
 
delete 'student' ,’95003’,’Sname:nickName’
 
deleteall 'student' ,’95003’   　(删除整个行记录)

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

/**
 * hbase只关注rowkey，column Family(列族)，并没有说在创建表的时候指定cq(列限定修饰符)有多少，这也是hbase列式存储的特点，
 *     所以在hbase API中是没有提供delete 一个列下的所有数据的
 * 
 *     同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
 * 1，Table table = connection.getTable(TableName.valueOf(tableName)); 先获取到表
 * 2，插入：(① 创建Put对象，② 然后通过方法 addColumn将列、列限定符、值 放到put对象，③ 最后将put对象put到表格)
 *     Put put = new Put(rowKey.getBytes());
 *  put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
 *  table.put(put);        
 * 3，删除：
 * Table table = connection.getTable(TableName.valueOf(tableName)); 同样首先获取到表
 * Delete delete = new Delete(rowKey.getBytes());    //通过传入行键，new一个删除对象    
 * //删除对象添加要被删除的列或列族                        
 * ① 删除指定列族的所有数据(此情况是列族下无列限定符时的情况)：delete.addFamily(colFamily.getBytes());        
 * ② 删除指定列的数据(此列主要说的是列限定修饰符)：delete.addColumn(colFamily.getBytes(), col.getBytes());
 * table.delete(delete); //最后就是表格delete掉 delete对象
 */
import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
 
public class Test_3 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
 
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    /**
     * 向某一行的某一列插入数据
     * 
     * @param tableName 表名
     * @param rowKey    行键
     * @param colFamily 列族名
     * @param col       列名(如果其列族下没有子列，此参数可为空)
     * @param val       值
     * @throws IOException
     */
    public static void insertRow(String tableName, String rowKey, String colFamily, String col, String val)throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Put put = new Put(rowKey.getBytes());
        put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
        table.put(put);
        table.close();
        close();
    }
 
    /**
     * 根据表名查找表信息
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner) {
            showCell((result));
        }
        close();
    }
    /**
     * 
      * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
    /**
     * 
     * 删除数据
     * 
     * @param tableName 表名
     * 
     * @param rowKey    行键
     * 
     * @param colFamily 列族名
     * 
     * @param col       列名
     * 
     * @throws IOException
     * 
     */
    public static void deleteRow(String tableName, String rowKey, String colFamily, String col) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Delete delete = new Delete(rowKey.getBytes());
        if(col == null) {
            //删除指定列族的所有数据(此情况是列族下无列限定符时的情况)
            delete.addFamily(colFamily.getBytes());
            table.delete(delete);
            table.close();
        }else {
            //删除指定列的数据(此列主要说的是列限定修饰符)
            delete.addColumn(colFamily.getBytes(), col.getBytes());
            table.delete(delete);
            table.close();
        }
        close();
    }
 
    public static void main(String[] args) {
        Test_3 t = new Test_3();
        boolean flag = true;
        while (flag){
            System.out.println("------------向已经创建好的表中添加和删除指定的列簇或列--------------------");
            System.out.println("              请输入您要进行的操作   1- 添加          2-删除                       ");
            Scanner scan = new Scanner(System.in);
            String choose1 = scan.nextLine();
            switch (choose1) {
            case "1":
                try {
                    //put 'student','95003','Sname','wangjinxuan' 　　　　(添加列)
                    //put 'student','95003','Sname:nickName','wang'　　　　(添加列族)
                    //put 'student','95003','Sname:firstName','jinxuan'  (添加列族)
//                    t.insertRow(tableName, rowKey, colFamily, col, val);
                    t.insertRow("student", "95003", "Sname",null, "wangjingxuan");
                    t.insertRow("student", "95003", "Sname", "nickName", "wang");
                    t.insertRow("student", "95003", "Sname", "firstName", "jingxuan");
                    System.out.println("插入成功：");
                    t.getData(tableName);
                } catch (IOException e) {
                    e.getMessage();
                }
                break;
            case "2":
                try {
                    System.out.println("----------------------删除前，表的原本信息如下---------------------");
                    t.getData(tableName);
                    //delete 'student' ,’95003’,’Sname’
                    //delete 'student' ,’95003’,’Sname:nickName’
//                    t.deleteRow(tableName, rowKey, colFamily, col);    
                    t.deleteRow("student", "95003", "Sname", "firstName");    
                    System.out.println("-----------------------删除成功-----------------------------\n");
                    System.out.println("---------------------删除后，表的信息如下---------------------");
                    t.getData(tableName);
                } catch (IOException e) {
                    e.getMessage();
                }
                break;
            }
            System.out.println(" 你要继续操作吗？ 是-true 否-false ");
            flag = scan.nextBoolean();
        }
        System.out.println("   程序已退出！    ");
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

清空指定的表的所有记录数据：

import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.util.Bytes;
public class Test_4 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    /**
     * 
     * 清空制定的表的所有记录数据
     * 
     * @param args
     * 
     * @throws IOException
     * 
     */
    public static void clearRows(String tableName) throws IOException {
        init();
        HBaseAdmin admin1 = new HBaseAdmin(configuration);
        // 读取了之前表的表名 列簇等信息，然后再进行删除操作。
        HTableDescriptor tDescriptor = admin1.getTableDescriptor(Bytes.toBytes(tableName));
        // 总思想是先将原表结构保留下来，然后进行删除，再重新依据保存的信息重新创建表。
        TableName tablename = TableName.valueOf(tableName);
        // 删除表
        admin.disableTable(tablename);
        admin.deleteTable(tablename);
        // 重新建表
        admin.createTable(tDescriptor);
        close();
    }
    /**
     * 
     * 根据表名查找表信息
     * 
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner){
            showCell((result));
        }
        close();
    }
 
    /**
     * 
     * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
 
    public static void main(String[] args) {
        Test_4 test_4 = new Test_4();
        Scanner scan = new Scanner(System.in);
        System.out.println("请输入要清空的表名");
        String tableName = scan.nextLine();
        try {
            System.out.println("表原来的信息：");
            test_4.getData(tableName);
            test_4.clearRows(tableName);
            System.out.println("表已清空：");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式

统计表的行数:

import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
public class Test_5 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    //建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    public static void countRows(String tableName) throws IOException{
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        int num = 0;
        for (Result result = scanner.next(); result != null; result = scanner.next()){
            num++;
        }
        System.out.println("行数：" + num);
        scanner.close();
        close();
    }
    public static void main(String[] args) throws IOException {
        Test_5 test_5 = new Test_5();
        Scanner scan = new Scanner(System.in);
        System.out.println("请输入要统计行数的表名");
        String tableName = scan.nextLine();
        test_5.countRows(tableName);
    }
}

熟悉常用的hbase操作,hbase,大数据,云计算,hadoop,分布式文章来源地址https://www.toymoban.com/news/detail-751726.html