Drill+IndexR性能测试

• 测试准备
  
  1. mqm项目业务数据。
  2. Hive表：mqm_test(Hive Text Format)
  3. Hive表：indexr_mqm_test(Hive IndexR Format)
• 测试工具
  
  1. hive beeline
  2. drill

• 测试结果

  1. 220W数据量

     • count

       1. select count(1) from mqm_test
          hive:46.43s, drill: 44.377s

       2. select count(1) from indexr_mqm_test
          hive:47.681s, drill: 48.713s

     • group/count/orderby

       1. select count(common_eventtype) from mqm_test group by common_eventtype
          hive:51.535s, drill: 14.366s

       2. select count(common_eventtype) from indexr_mqm_test group by common_eventtype
          hive:38.1s, drill: 1.578s

       3. select common_eventtype, common_eventdate, count(common_eventtype) from mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          hive: 66.256s, drill:23.856s

       4. select common_eventtype, common_eventdate, count(common_eventtype) from indexr_mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          hive: 66.256s, drill:7.128s

  2. 2500W数据量

     • group/count/orderby

       1. select count(common_eventtype) from mqm_test group by common_eventtype
          drill: 67.938s

       2. select count(common_eventtype) from indexr_mqm_test group by common_eventtype
          drill: 7.207s

       3. select common_eventtype, common_eventdate, count(common_eventtype) from mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:67.717s

       4. select common_eventtype, common_eventdate, count(common_eventtype) from indexr_mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:8.055s

  3. 7000W数据量

     • group/count/orderby

       1. select count(common_eventtype) from mqm_test group by common_eventtype
          drill: 250.636s

       2. select count(common_eventtype) from indexr_mqm_test group by common_eventtype
          drill: 14.881s

       3. select common_eventtype, common_eventdate, count(common_eventtype) from mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:252.804s

       4. select common_eventtype, common_eventdate, count(common_eventtype) from indexr_mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:24.791s

  4. 1.5亿数据量

     • group/count/orderby

       1. select count(common_eventtype) from mqm_test group by common_eventtype
          drill: 250.636s

       2. select count(common_eventtype) from indexr_mqm_test group by common_eventtype
          drill: 35.126s

       3. select common_eventtype, common_eventdate, count(common_eventtype) from mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:252.804s

       4. select common_eventtype, common_eventdate, count(common_eventtype) from indexr_mqm_test group by common_eventtype, common_eventdate order by common_eventdate desc
          drill:67.839s