Fork me on GitHub
Suzf  Blog

Tag couchbase

[译] Couchbase 使用 cbrestore 恢复数据

恢复使用 cbbackup 命令备份的 bucket 数据,使用 cbrestore 命令 恢复数据到一个 新的群集的bucket中。

当恢复数据的时候,你不得不根据你正在执行的还原类型选择一个合适的还原序列。恢复群集数据的方法 取决于备份群集的方式。

如果使用 cbbackup 备份 bucket 数据,你可以恢复数据到相同或者不同配置的群集中去。 这是因为cbbackup存储格式为bucket的数据的信息使其能够被恢复到新的群集上。

注意:如果数据是使用直接拷贝文件的方式备份的, 之后你恢复数据的时候必须恢复到相同群集中。

cbrestore 从使用 cbbackup 命令备份的数据中取信息存储到群集中。群集的配置不必匹配备份数据时所使用的。它可以在灾难恢复的情况下,将数据传送到一个新的群集中。或者是更新或者是扩展现有的群集版本。

因为数据可灵活恢复,它提供了许多不同的方案可以在已经备份的数据上执行:

恢复数据到不同大小和配置的群集中。
在相同 或不同配置的群集中 传输 或者 恢复数据到不同的 bucket。
恢复选定数据到 相同 或不同配置的群集一个不同的 bucket 中。

cbrestore 命令的基本用法:

cbrestore [options] [source] [destination]

Where:

[options]
    Options specifying how the information should be restored into the cluster. Common options include:

        --bucket-source

        Specify the name of the bucket data to be read from the backup data that will be restored.
        --bucket-destination

        Specify the name of the bucket the data will be written to. If this option is not specified, the data will be written to a bucket with the same name as the source bucket.
        --add

        Use --add instead of --set to avoid overwriting the existing items in the destination.

[source]
    The backup directory specified to cbbackup where the backup data was stored.

[destination]
    The REST API URL of a node within the cluster where the information will be restored.

cbrestore 命令一次只能恢复一个单独的bucket的数据。如果你备份了这个群集的bucket,则必须还原每个bucket到集群。所有目的bucket必须已经存在,因为 cbrestore 不会帮你要恢复的节点中创建配置bucket。

举个栗子
恢复一个单独的bucket 到群集中去:

cbrestore \
    /backups/backup-2012-05-10 \
    http://Administrator:[email protected]:8091 \
    --bucket-source=XXX
    [####################] 100.0% (231726/231726 msgs)
    bucket: default, msgs transferred...
    :                total |       last |    per sec
    batch :                  232 |        232 |       33.1
    byte  :             10247683 |   10247683 |  1462020.7
    msg   :               231726 |     231726 |    33060.0
    done

恢复bucket 到 群集中不同的 bucket 中:

cbrestore \
    /backups/backup-2012-05-10 \
    http://Administrator:[email protected]:8091 \
    --bucket-source=XXX \
    --bucket-destination=YYY
    [####################] 100.0% (231726/231726 msgs)
    bucket: default, msgs transferred...
    :                total |       last |    per sec
    batch :                  232 |        232 |       33.1
    byte  :             10247683 |   10247683 |  1462020.7
    msg   :               231726 |     231726 |    33060.0
    done

上面 msg 行统计的信息是恢复到群集中bucket 中文档的数量。

在恢复的时候过滤 key

cbrestore命令包括对于在备份过程中创建的文件过滤 Key 还原到数据库。
这是除了在备份过程中可用的过滤支持的另一方面。

该规范是以一个正则表达式的形式在cbrestore命令提供的一个参数。
例如,将以object开头的key的数据还原到一个bucket中:

cbrestore /backups/backup-20120501 http://HOST:8091 \
    -u Administrator -p password \
    -b default \
    -k '^object.*'
    2013-02-18 10:39:09,476: w0 skipping msg with key: sales_7597_3783_6
    ...
    2013-02-18 10:39:09,476: w0 skipping msg with key: sales_5575_3699_6
    2013-02-18 10:39:09,476: w0 skipping msg with key: sales_7597_3840_6
    [                    ] 0.0% (0/231726 msgs)
    bucket: default, msgs transferred...
    :                total |       last |    per sec
    batch :                    1 |          1 |        0.1
    byte  :                    0 |          0 |        0.0
    msg   :                    0 |          0 |        0.0
    done

只复制匹配到特定前缀的 keys 到默认的 bucket 中。对于跳过的每个键,提供了一个信息消息。
其余的输出显示转移的记录和正常的摘要。

通过拷贝文件恢复数据

要将数据恢复到相同配置的群集中,需要先关闭整个群集。当你恢复完数据之后,你需要将整个群集启动。在这种情况下,您正在用备份的数据文件的备份版本替换整个群集数据和配置,
然后用保存的群集文件的版本重新启动群集。
重要提示: 确保任何恢复的文件数组指定了适当的Couchbase用户。

当在同一个群集中恢复数据时,验证以下过程:
备份和恢复必须使用相同版本的 Couchbase。
整个群集必须包含相同数量的节点。
每个节点必须有相同的IP 或者 在备份的时候配置相同的主机名。
所有的 config.dat 配置文件以及所有数据库文件必须恢复到原来的位置。

完成还原过程所需的步骤是:
在所有节点上停止couchbase服务。
在每个节点上,从你的备份的文件中恢复数据库,stats.json,配置文件config.dat。
在每个节点上重启服务。

Parent topic: Backup and restore

源文: Restoring with cbrestore

 

[译] Couchbase 使用 cbbackup 备份

cbbackup 工具 是一个灵活的备份命令,使您可以备份本地数据和远程节点和涉及您的集群数据的不同组合:

单节点单buckets 单节点上所有buckets 整个群集上单个buckets 整个群集上所有buckets

备份可以在一个单独的节点直接拷贝文件,也可以通过连接远程群集然后通过数据流的形式保存到备份位置。备份可以运行在运行的节点或集群或脱机节点上执行。

cbbackup 命令可以以一种容易恢复的格式储存数据。当需要恢复的时候使用 cbrestore,你可以还原数据到任何配置的群集。源和目的群集不需要满足你使用 cbbackup 命令时保存的信息。

[译] Couchbase backup and restore

定期备份整个群集的数据,当需要恢复数据的时候可以最小化数据的不一致性。

备份你的数据应该是你的集群上定期的任务,以确保您不会在严重的硬件或安装失败的情况下丢失信息。

有许多用于执行备份的方法:

使用 cbbackup
cbbackup命令,可以在单个节点,单桶,或整个群集备份到一个灵活的备份架构,它可以将数据恢复到相同或不同的集群和水桶。所有的备份可以实时集群或节点上执行。该命令 cbbackup 是最灵活和推荐的备份工具。
使用 文件副本
正在运行的或脱机集群可以通过复制每个节点上的文件进行备份。使用这种方法,你只能还原到相同的配置群集。
注意:
由于Couchbase Server的灵活的特性,不可能创建整个集群的一个完整的即时备份和快照。因为数据总是被更新或修改,这是不可能采取正确的快照。
您必须备份和恢复整个集群,以尽量减少任何数据不一致。Couchbase是每个项目总是一致的,但不保证完全一致集群或顺序持久性。
 相关连接

[faq] Couchbase cant start because of the ip addr nightmare

许久之前,再次打开在本地 VM 创建的 Couchbase 有些异常 服务无法正常运行 查看日志,如下:
# /opt/couchbase/var/lib/couchbase/logs/info.log
[ns_server:warn,2016-08-15T13:47:59.536+08:00,[email protected]:dist_manager<0.129.0>:dist_manager:wait_for_address:121]Cannot listen on address `cb1.suzf.net`: eaddrnotavail
[ns_server:info,2016-08-15T13:47:59.536+08:00,[email protected]:dist_manager<0.129.0>:dist_manager:wait_for_address:125]Configured address `cb1.suzf.net` seems to be invalid. Giving OS a chance to bring it up.
[ns_server:error,2016-08-15T13:48:00.537+08:00,[email protected]:dist_manager<0.129.0>:dist_manager:init:178]Configured address `cb1.suzf.net` seems to be invalid. Will refuse to start for safety reasons.
看起来是说主机名称  cb.suzf.net  无效,以至于服务 无法正常启动。 O 想起来了, 曾经修改过 /etc/hosts 记录
#grep cb /etc/hosts
172.16.9.10     cb1.suzf.net
172.16.9.20     cb2.suzf.net
而当前的IP为
#ifconfig eth1|grep 'inet addr'
          inet addr:172.16.9.11  Bcast:172.16.9.255  Mask:255.255.255.0
下面来更正 hosts 记录
#grep cb /etc/hosts
172.16.9.11     cb1.suzf.net
172.16.9.20     cb2.suzf.net
It worked.

There are several ways you can provide hostnames:

  • When installing a Couchbase Server on a machine.
  • When adding a node to an existing cluster for an online upgrade.
  • Using a REST API call.

Hostname Errors

Provide the hostname and port for the node and administrative credentials for the cluster. The value you provide for hostname must be a valid hostname for the node. Possible errors that can occur:

  • Could not resolve the host name. The host name you provide as a parameter does not resolve to a IP address.
  • Could not listen. The host name resolves to an IP address, but no network connection exists for the address.
  • Could not rename the node because name was fixed at server start-up.
  • Could not save address after rename.
  • Requested name host name is not allowed. Invalid host name provided.
  • Renaming is disallowed for nodes that are already part of a cluster.
节选自: http://developer.couchbase.com/documentation/server/4.1/install/hostnames.html

Couchbase CLI Notes

cbreset_password

Syntax

The basic syntax to reset the administrative password:

cbreset_password [hostname]:[port]

#./bin/cbreset_password 127.0.0.1:8091
Please enter the new administrative password (or <Enter> for system generated password):
Running this command will reset administrative password.
Do you really want to do it? (yes/no)yes
Resetting administrative password...
Password for user Administrator was successfully replaced.

未完待续  ...

 

How-to: install couchbase on linux

上一篇我们介绍了 What is Couchbase Server 接下来我们来看如获取安装包以及简单地配置 官方站点: http://www.couchbase.com 文档归档: http://developer.couchbase.com/documentation-archive 有两个版本:企业版和社区版 这里以社区版为例

What is Couchbase Server

Couchbase Server Screenshot.jpg
Developer(s) Couchbase, Inc.
Stable release 3.0.3 / March 28, 2015
Written in C++, Erlang, C[1]
Operating system Cross-platform
Type distributed key-value / document database system
License Apache License (Open Source edition), Proprietary (Free Community edition and Paid Enterprise edition)
Website www.couchbase.com

Couchbase Server, originally known as Membase, is an open-source, distributed (shared-nothing architecture) NoSQL document-oriented database that is optimized for interactive applications. These applications must serve many concurrent users by creating, storing, retrieving, aggregating, manipulating and presenting data. In support of these kinds of application needs, Couchbase is designed to provide easy-to-scale key-value or document access with low latency and high sustained throughput. It is designed to be clustered from a single machine to very large-scale deployments spanning many machines.

Couchbase Server provides on-the-wire client protocol compatibility with memcached,[2] but is designed to add disk persistence, data replication, live cluster reconfiguration, rebalancing and multitenancy with data partitioning.

In the parlance of Eric Brewer’s CAP theorem, Couchbase is a CP type system meaning it provides consistency and partition tolerance. However Couchbase Server can be set up as an AP system with multiple clusters using XDCR (Cross Data Center Replication).

Contents

Product history

Membase was developed by several leaders of the memcached project, who had founded a company, NorthScale, to develop a key-value store with the simplicity, speed, and scalability of memcached, but also provided the storage, persistence and querying capabilities of a database. The original membase source code was contributed by NorthScale, and project co-sponsors Zynga and NHN to a new project on membase.org in June 2010.

On February 8, 2011, the Membase project founders and Membase, Inc. announced a merger with CouchOne (a company with many of the principal players behind CouchDB) with an associated project merger. The merged company was called Couchbase, Inc. In January 2012, Couchbase released Couchbase Server 1.8. On December 2012, Couchbase Server 2.0 was released, with new features including a new JSON document store, indexing and querying, incremental MapReduce and cross datacenter replication.[3]

High-level architecture

Every Couchbase node is architecturally identical consisting of a data manager and cluster manager component.

Cluster manager

The cluster manager supervises the configuration and behavior of all the servers in a Couchbase cluster. It configures and supervises internode behavior like managing replication streams and rebalancing operations. It also provides metric aggregation and consensus functions for the cluster, and a RESTful cluster management API. The cluster manager is built atop Erlang/OTP, a proven environment for building and operating fault-tolerant distributed systems.

Replication and failover

  • Multi-model replication support: Peer-to-peer replication support with underlying architecture supporting master-slave replication
  • Configurable replication count: Balance resource utilization with availability requirements
  • High-speed failover: Fast failover to replicated items based upon request
  • XDCR: Cross Data Centre Replication [4]

Data manager

The data manager is responsible for storing and retrieving documents in response to data operations from applications.

  • Asynchronously writes data to disk after acknowledging write to client. In version 1.7 and later, applications can ensure data is synced to more than one server, while disk writes are still asynchronous.
    • Tunables to define item ages that affect when data is persisted.
  • Supports working set greater than a memory quota per "node" or "bucket"
    • Tunables to affect how max memory and migration from main-memory to disk is handled.
  • Configurable “tap” interface: External systems can subscribe to filtered data streams supporting, for example, full text search indexing, data analytics or archiving.[5]

Data format

A document is the most basic unit of data manipulation in Couchbase Server. Documents are stored in JSON document format with no predefined schemas.

Object-managed cache

Couchbase Server includes a built-in multithreaded object-managed cache that implements memcached compatible APIs such as get, set, delete, append, prepend etc.

Storage engine design

Couchbase Server has a tail-append storage design that is immune to data corruption, OOM killers or sudden loss of power. ["immune" to sudden loss of power is a claim that would need to be substantiated or documented, since appending to a file could still result in loss of data if power was interrupted in the middle of a disk write operation] Data is written to the data file in an append-only manner, which enables Couchbase to do mostly sequential writes for update, and provide an optimized access patterns for disk I/O.

Performance

Cisco published a benchmark that measures the latency and throughput of Couchbase Server [6] with a mixed workload. Another performance benchmark done by Altoros, compares Couchbase Server with other NoSQL database solutions.[7]

Licensing and support

Couchbase Server is a packaged version of Couchbase's open source technology and is available in two variants: a Community Edition without recent bug fixes as Open Source (Apache 2.0 license[8]) distribution, and an Enterprise Edition for commercial use.[9]

Couchbase Server builds are available for Ubuntu, Debian, Red Hat, Windows and Mac OS X platforms.

See also

Bibliography

References

 

 

  1. "Couchbase Server Editions". Couchbase.

External links