HBASE migrate table part 1

December 19, 2014 bigdata, hbase No comments

Copy table between different hbase clusters – Version 0.96.2-hadoop2

Create test table and initial some records.

$hbase shell
2014-12-19 15:01:36,085 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help&lt[......]

Read more

MySQL-Oslayer-Performance-Optimization

December 13, 2014 Architecture, MYSQL, performance No comments

upload on 2014.12

important tips of MySQL database design for better performance

Read : mysql-system-opt[……]

Read more

MySQL message subscription system

December 9, 2014 Architect, Architecture, MYSQL, replication, software No comments

Introducing MQ system based on MySQL (canal + roma)
Using this system to subscribe message between different platforms.

View this PDF roma_system
Also download this PDF from slideshare[……]

Read more

MyAWR : MySQL Workload Report V3 release

December 2, 2014 Architecture, MYSQL, performance No comments

myawr-v3 release now !

history of myawr:

————————–
v1: create by noodba

mysql load information and system information 

v2: create by louis liu

myawr_v2 add :

1 more os information module.

2 multiple instance support (extend table by db_port)

3 modified[......]

Read more

webm_v2 has been released

October 21, 2014 Architect, NoSQL, rdbms, software No comments

Webm_v2 was add oracle and mongodb monitor module

webm was agent uploaded mode to save statistics

we use mysql database as monitor server

we already design tables to store these data (consider for future analysis)

you can download from github : https://github.com/ylouis83/webm[……]

Read more

Webm: MySQL database performance web monitor

September 28, 2014 Architect, MYSQL, performance No comments , ,

You can download webm from github: https://github.com/ylouis83/webm

webm: mysql web key performance monitor

webm is a tool that display key value graph on website and webm was developed by javascript and mysqlmon ( mysql data collection tool wrote by AnySQL)

Environment need:

Linux vers[……]

Read more

关于Dell 推出第13代服务器的一些想法

September 12, 2014 Architect, hardware No comments

戴尔近日推出了旗下的13G服务器,其主力机型为R730xd,包含了诸多的特性,为其成为主流db server以及规模存储集群打下了良好的基础。

具体参考:http://www.storagereview.com/dell_poweredge_13g_r730xd_review
http://www.storagereview.com/dell_poweredge_gen13_servers_released

https://www.youtube.com/watch?v=YSGzB7umHa0

具体增强为:

1.CPU 为intel haswall最新架构,减少了功能[……]

Read more

HBASE Performance Test by YCSB

September 3, 2014 bigdata, hbase No comments

Read this PDF:Hbase_performance[……]

Read more

关于Partial write for all MySQL and Oracle

September 2, 2014 Internals, MYSQL, oracle No comments

很久之前看过这个帖子,当时没有细究,这几天看了下mysql相关的internal 突然想起这个问题,跟三四个朋友讨论了下 想把这个问题展开讲一讲。

源帖子内容见:https://community.oracle.com/thread/1087650?start=0&tstart=0

这个帖子的内容为一个老外问oracle如何避免partial write的,也就是大家所说的半个IO问题,IO 在写到一半的时候发生断裂,(瞬间断线,瞬间crash瞬间存储或者机器故障等) 虽然这个情况极少发生,但是在非常规恢复的时候也是经常遇到的,例如某个block内部出现不一致,也就是我们所谓的逻辑[……]

Read more

Some data collection display

August 21, 2014 MYSQL, performance No comments , , ,

Reference Oracle AWR. These two was developed by python and perl.

Both of these are following data collection method.

Use these tools can help quick identify problem.

MyAWR v2 –collection of mysql and os information

MongoAWR v1 — collection of mongodb and os information[……]

Read more