블로그 이미지


1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30


2011. 8. 29. 20:51 Brain Trainning/NoSQL

Zachary Witte:

When you have the instance basically set, go back into the AWS control panel, right click the instance and choose Create Image. You can start up any number of these for the replica set, but you need to change the /etc/hostname and /etc/hosts file to reflect the individual IP address and hostname of the bot (db1, db2, db3, etc.)


Before you set up MongoDB on EC2 make sure you understand the various aspects of running MongoDB in the Amazon cloud:


posted by LifeisSimple
2011. 6. 17. 14:46 Brain Trainning/NoSQL
신규 시스템을 도입하면 그것도 핵심이 되는 서비스에 도입을 하게 된다면 보수적인 입장에서 많이 고민해야할 듯 합니다. 
MongoDB의 경우도 수시로 마이너 업데이트가 진행될만큼 아직 안전성면에서 잘 되어 있다고 장담하기 힘듭니다. 
또한 각종 써드파티 툴들의 경우도 마찬가지인데 MSSQL 계통을 사용했다면 불편한 점이 많고 부수적인 개발이 많이 들어갑니다. 

제일 중요한 점은 역시나 레퍼런스가 부족하고 기술지원이 어렵기에 쉽사리 할수있다 라는 무모한 생각으로 접근하면 대규모 서비스에 있어서 낭패를 볼 수 있습니다. 정식 서비스보다는 파일럿 성격 혹은 몇번의 다운과 갈아엎었을때 부담이 적은 경우에 선 도입한 후 어느 정도 운영 기술이 축적된 이후에 추천합니다.
출처 : http://highscalability.com/blog/2011/6/15/101-questions-to-ask-when-considering-a-nosql-database.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+HighScalability+%28High+Scalability%29&utm_content=Google+International

101 Questions To Ask When Considering A NoSQL Database

You need answers, I know, but all I have here are some questions to consider when thinking about which database to use. These are taken from my webinar What Should I Do? Choosing SQL, NoSQL or Both for Scalable Web Applications. It's a companion article to What The Heck Are You Actually Using NoSQL For?

Actually, I don't even know if there are a 101 questions, but there are a lot/way too many. You might want to use these questions as kind of a NoSQL I Ching, guiding your way through the immense possibility space of options that are in front of you. Nothing is fated, all is interpreted, but it might just trigger a new insight or two along the way.

Where Are You Starting From?

  • A can do anything green field application?
  • In the middle of a project and worried about hitting bottlenecks? 
  • Worried about hitting the scaling wall once you deploy?
  • Adding a separate loosely coupled service to an existing system?
  • What are your resources? expertise? budget?
  • What are your pain points? What's so important that if it fails you will fail? What forces are pushing you?
  • What are your priorities? Prioritize them. What is really important to you, what must get done?
  • What are your risks? Prioritize them. Is the risk of being unavailable more important than being inconsistent?

What Are You Trying To Accomplish?

  • What are you trying to accomplish? 
  • What's the delivery schedule? 
  • Do the research to be specific, like Facebook did with their messaging system:
Facebook chose HBase because they monitored their usage and figured out what was needed: a system that could handle two types of data patterns.

Things To Consider...Your Problem

  • Do you need to build a custom system?
  • Access patterns: 1) A short set of temporal data that tends to be volatile 2) An ever-growing set of data that rarely gets accessed 3) High write loads 4) High throughput, 5) Sequential, 6) Random
  • Requires scalability?
  • Is availability more important than consistency, or is it latency, transactions, durability, performance, or ease of use?
  • Cloud or colo? Hosted services? Resources like disk space?
  • Can you find people who know the stack?
  • Tired of the data transformation (ORM) treadmill? 
  • Store data that can be accessed quickly and is used often?
  • Would like a high level interface like PaaS?  

Things To Consider...Money

  • Cost? With money you have different options than if you don't. You can probably make the technologies you know best scale.
  • Inexpensive scaling?
  • Lower operations cost? 
  • No sysadmins?
  • Type of license?
  • Support costs?

Things To Consider...Programming

  • Flexible datatypes and schemas?
  • Support for which language bindings?
  • Web support: JSON, REST, HTTP, JSON-RPC
  • Built-in stored procedure support? Javascript?
  • Platform support: mobile, workstation, cloud
  • Transaction support: key-value,  distributed, ACID, BASE, eventual consistency, multi-object ACID transactions.
  • Datatype support: graph, key-value, row, column, JSON, document, references, relationships, advanced data structures, large BLOBs.
  • Prefer the simplicity of transaction model where you can just update and be done with it? In-memory makes it fast enough and big systems can fit on just a few nodes.

Things To Consider...Performance

  • Performance metrics: IOPS/sec, reads, writes, streaming?
  • Support for your access pattern: random read/write; sequential read/write; large or small or whatever chunk size you use. 
  • Are you storing frequently updated bits of data? 
  • High Concurrency vs High Performance?
  • Problems that limit the type of work load you care about?
  • Peak QPS on highly-concurrent workloads?
  • Test your specific scenarios?

Things To Consider...Features

  • Spooky scalability at a distance: support across multiple data-centers?
  • Ease of installation, configuration, operations, development, deployment, support, manage, upgrade, etc.
  • Data Integrity: In DDL, Stored Procedure, or App
  • Persistence design: Memtable/SSTable; Apend-only B-tree; B-tree; On-disk linked lists; In-memory  replicated; In-memory snapshots; In-memory only; Hash; Pluggable.
  • Schema support: none, rigid, optional, mixed
  • Storage model: embedded, client/server, distributed, in-memory
  • Support for search, secondary indexes, range queries, ad-hoc queries, MapReduce?
  • Hitless upgrades?

Things To Consider...More Features

  • Tunability of consistency models?
  • Tools availability and product maturity?
  • Expand rapidly? Develop rapidly? Change rapidly?
  • Durability? On power failure?
  • Bulk import? Export? 
  • Hitless upgrades?
  • Materialized views for rollups of attributes?
  • Built-in web server support?
  • Authentication, authorization, validation?
  • Continuous write-behind for system sync?
  • What is the story for availability, data-loss prevention, backup and restore?
  • Automatic load balancing, partitioning, and repartitioning?
  • Live addition and removal of machines?

Things To Consider...The Vendor

  • Viability of the company? 
  • Future direction?
  • Community and support list quality?
  • Support responsiveness?
  • How do they handle disasters?
  • Quality and quantity of partnerships developed?
  • Customer support: enterprise-level SLA, paid support, none 
posted by LifeisSimple
2011. 4. 4. 15:52 Brain Trainning/NoSQL

Auto Increment with MongoDB

We are currently working on an app that uses a number of technologies, including PHP, Python, and MongoDB. Recently, a need arose to use sequential identifiers for users, similar to an auto_increment column in MySQL.

If you've used MongoDB, you might be familiar with the default behavior of using a UUID as the primary key. This is convenient, especially if you partition your database across servers, because you don't have to coordinate the primary key in any way. If you use sequential identifiers (as I demonstrate in this post), you can use multiple servers and interleave identifiers by advancing each server's sequence by the total number of servers. (For example, with two servers, advance each sequence by two, so one server generates even identifiers, and the other generates odd.)

I'd rather not discuss the advantages and disadvantages of either approach, because it's exactly this debate that makes it very difficult to find any useful information on using sequential identifiers with MongoDB. Instead, I'm just going to explain how I did it, and hope this is helpful to someone. :-)

First, create a sequence collection that you can use to determine the next identifier in the sequence. The following creates a collection called seq that has a single sequence in it (for users), but you can add as many as you need:


  1. db.seq.insert({"_id":"users", "seq":new NumberLong(1)});

If you assign seq to 1 instead of new NumberLong(1), it will be interpreted as a float due to a JavaScript quirk.

Before adding a new user, you need to increment the sequence by one and fetch the next identifier. Fortunately, the findandmodify() command provides an atomic way to do this. Using the MongoDB shell, the command would look something like this:


  1. db.seq.findAndModify({
  2. query: {"_id":"users"},
  3. update: {$inc: {"seq":1}},
  4. new: true
  5. });

Because I'm using Lithium, I added a method for fetching the next identifier to myUser model:


  1. <?php
  3. namespace app\models;
  5. class User extends \lithium\data\Model {
  7. static public function seq() {
  8. $seq = static::_connection()->connection->command(
  9. array('findandmodify' => 'seq',
  10. 'query' => array('_id' => 'users'),
  11. 'update' => array('$inc' => array('seq' => 1)),
  12. 'new' => TRUE
  13. )
  14. );
  16. return $seq['value']['seq'];
  17. }
  19. }
  21. ?>

If you're not using Lithium, you can use the MongoDB class to execute a command().

With this in place, adding a new user is a simple process. I create an array called$data with everything I want to store for a user, and then do the following:


  1. <?php
  3. $user = User::create($data);
  4. $user->_id = User::seq();
  5. $success = $user->save();
  7. ?>

This example should be easy to adapt to any environment. Once you have the next identifier in the sequence, you simply store it as you would any other data.

I hope to blog more about both MongoDB and Lithium. As these technologies are still pretty new to me, please feel free to point out any improvements. I'll update the post accordingly.

posted by LifeisSimple
2011. 3. 31. 17:53 Brain Trainning/NoSQL
Replication Set 에서 DB 커넥팅 방법

                  public void testFindAll() throws UnknownHostException, MongoException {

                                   // TODO Auto-generated method stub

                                   ArrayList<ServerAddress> addrs = new ArrayList<ServerAddress>();

                   addrs.add(new ServerAddress("", 27017));

                   addrs.add(new ServerAddress("", 27017));

                   addrs.add(new ServerAddress("", 27017));

                   Mongo connection = new Mongo(addrs);


                                   DB mongoJUnit = connection.getDB("deptDB");

                                   DBCollection dept = mongoJUnit.getCollection("deptJUnit");


                                   DBCursor cur = dept.find();


                                   int index = 1;

                                   for ( Iterator<DBObject> it = cur.iterator(); it.hasNext(); ++index) {

                                                     DBObject obj = it.next();


                                                     System.out.println("data("+ index+"):"+obj.toString());





posted by LifeisSimple
2011. 3. 31. 14:02 Brain Trainning/NoSQL
대략 이런식으로 Collection을 구성하면 괜찮지 않을지.... 


"_id" : ObjectId("4bfe4946cfbfb01420000001"),

"created_at" "Thu, 27 May 2010 10:25:46 +0000",

"profile_image_url" :"http://a3.twimg.com/profile_images/600304197/Snapshot_2009-07-26_13-12-43_normal.jpg",

"from_user" "drearyclocks",

"text" "Does anyone know who has better coverage, Optus or Vodafone? Telstra is still too expensive.",

"to_user_id" null,

"metadata" : {

"result_type" "recent"


"id" : {

"floatApprox" : 14825648892


"geo" null,

"from_user_id" : 6825770,

"search_term" "telstra",

"iso_language_code" "en",

"source" "&lt;a href=&quot;http://www.tweetdeck.com&;quot; rel=&quot;nofollow&quot;&gt;TweetDeck&lt;/a&gt;"


'Brain Trainning > NoSQL' 카테고리의 다른 글

[MongoDB] Auto Increment with MongoDB  (0) 2011.04.04
[MongoDB] Replication Set Connection 방법  (0) 2011.03.31
[MongoDB] Config 파일 옵션  (0) 2011.03.09
[MongoDB] Java Code Sample  (0) 2011.02.28
[MongoDB] MSSQL 2 MongoDB  (0) 2011.02.25
posted by LifeisSimple
2011. 3. 9. 18:11 Brain Trainning/NoSQL

출처 : www.mongodb.org

File Based Configuration

In addition to accepting Command Line Parameters, MongoDB can also be configured using a configuration file. A configuration file to use can be specified using the -f or --config command line options. On some packaged installs of MongoDB (for example Ubuntu & Debian) the default file can be found in /etc/mongodb.conf which is automatically used when starting and stopping MongoDB from the service.

The following example configuration file demonstrates the syntax to use:

# This is an example config file for MongoDB.
dbpath = /var/lib/mongodb
bind_ip =
noauth = true # use 'true' for options that don't take an argument
verbose = true # to disable, comment out.
Unfortunately flag parameters like "quiet" will register as true no matter what value you put after them. So don't write this: "quiet=false", just comment it out, or remove the line.


Basic database configuration

dbpath Location of the database files dbpath=/var/lib/mongodb
port Port the mongod will listen on port=27017
Specific IP address that mongod will listen on
logpath Full filename path to where log messages will be written logpath=/var/log/mongodb/mongodb.log
logappend Whether the log file will be appended (TRUE) or over-written (FALSE) logappend=true


cpu Enable periodic logging (TRUE) of CPU utilization and I/O wait cpu = true
verbose Verbose logging output verbose=true


noauth Turn authorization on/off. Off is currently the default noauth = true
auth Turn authorization on/off. Off is currently the default auth=false

Administration & Monitoring

nohttpinterface Disable the HTTP interface. The default port is 1000 more than the dbport nohttpinterface = true
noscripting Turns off server-side scripting. This will result in greatly limited functionality noscripting = true
notablescan Turns off table scans. Any query that would do a table scan fails. notablescan = true
noprealloc Disable data file preallocation. noprealloc = true
nssize Specify .ns file size for new databases in MB nssize = 16
mms-token Account token for Mongo monitoring server. mms-token=mytoken
mms-name Server name for Mongo monitoring server. mms-name=monitor.example.com
mms-interval Ping interval for Mongo monitoring server in seconds. mms-interval=15 
quota Enable quota management quota = true
quotaFiles Determines the number of files per Database (default is 8) quotaFiles=16


master In replicated mongo databases, specify here whether this is a slave or master master = true
slave In replicated mongo databases, specify here whether this is a slave or master slave = true
source Specify the source = master.example.com
only Slave only: specify a single database to replicate only = master.example.com
pairwith Address of a server to pair with. pairwith = master.example.com:27017
arbiter Address of arbiter server arbiter = aribiter.example.com:27018
autoresync Automatically resync if slave data is stale autoresync
opIdMem Size limit for in-memory storage of op ids in Bytes opIdMem=1000
fastsync Indicate that this instance is starting from a dbpath 
snapshot of the repl peer
Replica Sets
replSet Use replica sets with the specified logical set name.  Typically the optional seed host list need not be specified. replSet=<setname>[/<seedlist>]
oplogSize Custom size for replication operation log in MB. oplogSize=100


shardsvr Indicates that this mongod will participate in sharding *Optional and just changes the default port shardsvr=true

'Brain Trainning > NoSQL' 카테고리의 다른 글

[MongoDB] Replication Set Connection 방법  (0) 2011.03.31
[MongoDB] 트위터 Collections...  (0) 2011.03.31
[MongoDB] Java Code Sample  (0) 2011.02.28
[MongoDB] MSSQL 2 MongoDB  (0) 2011.02.25
[MongoDB] Comparing MongoDB java frameworks  (0) 2011.02.23
posted by LifeisSimple
2011. 2. 28. 17:56 Brain Trainning/NoSQL
-- 페이지 아래쪽에 코드 셈플있음... 

Mongo Java Driver 코드 셈플... 강한 에러를 동반할 수 있으니 잘 수정해서 사용해야 합니다.

package com.Mongo.Test;


import com.mongodb.*;

import org.junit.Before;

import org.junit.Test;

import static org.junit.Assert.*;


public class MongoDBUnitTestII {

    private DBCollection customers;



    public void setUp() throws Exception {

        Mongo mongo = new Mongo();

        DB db = mongo.getDB("webshop");

        customers = db.getCollection("customer");

        customers.remove(new BasicDBObject());




    public void shouldWorkWithMongoDB() {

        DBObject customerMaxMustermann = new BasicDBObject()

                .append("id", "4711")

                .append("name","Max Mustermann")


                .append("numberOfOrders", 3);


        DBObject savedCustomer = customers.findOne();


        assertEquals("4711", savedCustomer.get("id"));



   /* {"id" : 1234, "name": "Otto Normal", city: "Berlin",#

      numberOfOrders: 4, bankData: {accountNumber : "9876543210", bankCode : "30020011", accountHolder : "Otto Normal"}*/


    private void insertCustomerOttoNormal() {

        Customer customer = new Customer();


        customer.setName("Otto Normal");




        BankData bankData = new BankData();



        bankData.setAccountHolder("Otto Normal");








    public void shouldSaveAndFindCustomers() {



        // attention: all field names here begin with a capital letter

        DBCursor result = customers.find(new BasicDBObject("City", "Berlin"));


        Customer savedCustomer = (Customer) result.next();

        assertEquals("1234", savedCustomer.getId());

        assertEquals("30020011", savedCustomer.getBankData().getBankCode());




    public void shouldIncrementTheNumberOfOrdersIfTheCustomerPlaceAnOrder() {



        customers.update(new BasicDBObject("Id", "1234"),

                new BasicDBObject("$inc",

                        new BasicDBObject("NumberOfOrders", 1)));

        Customer savedCustomer = (Customer) customers

                .findOne(new BasicDBObject("NumberOfOrders", new BasicDBObject("$gt", 4)));


        assertEquals("Otto Normal", savedCustomer.getName());

        assertEquals(5, savedCustomer.getNumberOfOrders());




'Brain Trainning > NoSQL' 카테고리의 다른 글

[MongoDB] 트위터 Collections...  (0) 2011.03.31
[MongoDB] Config 파일 옵션  (0) 2011.03.09
[MongoDB] MSSQL 2 MongoDB  (0) 2011.02.25
[MongoDB] Comparing MongoDB java frameworks  (0) 2011.02.23
[MongoDB] 당신의 모든 Backup  (0) 2011.01.25
posted by LifeisSimple
2011. 2. 25. 12:26 Brain Trainning/NoSQL

import java.net.UnknownHostException;

import java.util.Calendar;

import java.util.GregorianCalendar;

import java.util.Iterator;

import java.util.List;

import java.util.regex.Pattern;

import org.junit.Test;

import java.sql.*;


import com.mongodb.BasicDBObject;

import com.mongodb.DB;

import com.mongodb.DBCollection;

import com.mongodb.DBCursor;

import com.mongodb.DBObject;

import com.mongodb.Mongo;

import com.mongodb.MongoException;



Public Class MongoDB {


       // MSSQL 연결

       public void testMSSQLConnection() throws Exception {

             // TODO Auto-generated method stub

             Connection conSQL = null;

             Statement stmt = null;

             ResultSet rs = null;


             // MSSQL

             String strJDBCDrv = "com.microsoft.sqlserver.jdbc.SQLServerDriver";

             String strJDBCUrl = "jdbc:sqlserver://localhost:1433;DatabaseName=SQLDB";

             String strQuery = "select testID, testName from T_Test order by testID desc";


             // MongoDB

             Mongo connection = new Mongo("MongoDBIP", 27017);

             DB mongoJUnit = connection.getDB("testDB");

             DBCollection dept = mongoJUnit.getCollection("T_Test");





                    System.err.println("드라이버 검색 성공");


             catch(ClassNotFoundException ee) {


                    System.err.println("드라이버 검색 실패");



             try {


                    conSQL = DriverManager.getConnection(strJDBCUrl, "sa", "1231231");


                    stmt = conSQL.createStatement();

                    rs = stmt.executeQuery(strQuery);


                    BasicDBObject putDB = new BasicDBObject();

                    //GregorianCalendar cal = new GregorianCalendar();



                           // System.out.println(rs.getString(1));


                           putDB.put("testID", rs.getInt(1));

                           putDB.put("testName", rs.getString(2));







             catch (Exception e){



             finally {







'Brain Trainning > NoSQL' 카테고리의 다른 글

[MongoDB] Config 파일 옵션  (0) 2011.03.09
[MongoDB] Java Code Sample  (0) 2011.02.28
[MongoDB] Comparing MongoDB java frameworks  (0) 2011.02.23
[MongoDB] 당신의 모든 Backup  (0) 2011.01.25
[MongoDB] MongoDB Cacti Plugin Templates  (0) 2011.01.21
posted by LifeisSimple
2011. 2. 23. 17:21 Brain Trainning/NoSQL

출처 : http://blog.knuthaugen.no/2010/07/comparing-mongodb-java-frameworks.html

Comparing MongoDB java frameworks

| 4 Comments | No TrackBacks

I while ago I gave a talk at a Oslo JavaBin meeting on NoSQL and in the research for that I wounded up writing some sample code for three different frameworks for using MongoDB from Java. The code is available on GitHub. The three are the 10Gen mongo driver (and not really a framework), morphia and mungbean.

The Whole Dynamic Thing

MongoDB is a schema-free document database and does not impose any restriction on what you can store in a collection. There is nothing stopping you from puttting entirely different objects into the same collection and thus nothing stopping you from omitting fields in two different objects of the same kind either. So how does this play with Javas strong static typing compared to a dynamic language like Ruby?

Dude, Where's My ORM?

Developers used to relational databases have been using ORMs for a long time to abstract away the dirty details of dealing with the database and separating the domain from the persistence. So, are any of the three frameworks an ORM for MongoDB? Well, no. But a more interesting question is: Do I need one? MongoDB is a very different beast than oracle/mysql/MSsql/PostgreSQL and different beasts need different handling. You need an abstraction model for MongoDB, not necessarily the ORM abstraction model.

MongoDB Java Driver

10Gen java driver basically gives you to options when storing data in MongoDB: Either subclassBasicDBObject which is the general database object, or implement the DBObject interface. Both approaches gives you the interface of Map from the standard java library and lets you put and get keys and values. The drawback is that there are a lot of methods in this interface and it will soon become tedious to implement them all for different domain objects. This it the DBObject interface methods:

public Object put(String s, Object o) { }
public void putAll(DBObject dbObject) { }
public void putAll(Map map) { }
public Object get(String s) { }
public Map toMap() { }
public Object removeField(String s) { }
public boolean containsKey(String s) { }
public boolean containsField(String s) { }
public Set<string> keySet() { }
public void markAsPartialObject() { }
public boolean isPartialObject() {}

So I'd prefer to subclass BasicDBObject instead or wrap it.

So how does code using it look like? Assume the very minimal domain objects Person andAddress

package no.kh.mongo.direct;
import com.mongodb.BasicDBObject;
public class Person extends BasicDBObject {
public Person () { }
public Person(String fullName, Address newAddress) {
put("name", fullName);
put("address", newAddress);
package no.kh.mongo.direct;
import com.mongodb.BasicDBObject;
public class Address extends BasicDBObject {
public Address() {
public Address(String streetName, String postalCode, String place, String country) {
put("street", streetName);
put("postalcode", postalCode);
put("place", place);
put("country", country);
public String place(){
return (String) get("place");

Using this class is done like this in the form of functional/integration tests, since they touch the database but written in Junit:

package no.kh.mongo.direct;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.Mongo;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.net.UnknownHostException;
import static junit.framework.Assert.assertEquals;
import static junit.framework.Assert.assertNotNull;
import static org.junit.Assert.assertNull;
public class PersonPersistance {
DB testDb;
DBCollection persons;
public void setUp() throws UnknownHostException {