bigdata

Presto-engine Privileges Control – Architect

January 20, 2022 Architect, bigdata, linux, system No comments

We use presto as the main data query engine to provide data query services for business parties. In our architecture, presto as a core capability requires access to data query permission control, including permission control at the table and column levels. We designed The following scheme to achieve this requirement

[……]

Read more

Clickhouse Materialized view

April 24, 2021 Architect, bigdata No comments

We are now using Clickhouse for OLTP  streaming computing system.

Read this PDF for some tips for MV of Clickhouse CK-MV[……]

Read more

Logcenter Project architect

August 16, 2017 Architect, Architecture, bigdata, hadoop, hive, network, NoSQL, rdbms No comments

We created a project called LC (log center) for ops department
All member of ops are using this system for analyzing in a lower layer.
We collects[……]

Read more

HBASE migrate table part 1

December 19, 2014 bigdata, hbase No comments

Copy table between different hbase clusters – Version 0.96.2-hadoop2

Create test table and initial some records.

$hbase shell
2014-12-19 15:01:[......]

Read more

HBASE Performance Test by YCSB

September 3, 2014 bigdata, hbase No comments

Read this PDF:Hbase_performance[……]

Read more