MongoDB Bulk Data Import From CSV

Background

mongoimport is a very handy tool for bulk data import. For the most simplest form of importing data in your collection is is the only built in tool available. Alternatively for import one can built an utility .At present this tool as limited capability and can import data from Extended JSON, CSV, or TSV only.

For your help, I have provided an fully functional data import file with 1.3 million+ records. You can download it from here and use as you like.

Using Import Utility

The process of data importing is very easy. Let me elaborate the process by steps.

Create a *.csv file for import where column/header names are at first row.

1) Place the import data file under MONGO_ROOT/BIN/

2) From shell navigate to bin under MONGO installation directory

3) You have to execute following command

mongoimport -d [DB_NAME] -c tmpprospect --type csv --file 
[IMPORT_FILE_NAME].csv --headerline

If you take the import file I provided, command looks like this,
where DB_NAME is “hawk” & [IMPORT_FILE_NAME] is “hawk-prospect-data”

mongoimport -d hawk -c tmpprospect --type csv 
--file hawk-prospect-data.csv --headerline

4) You will see mongoimport utility will split the import file into small chunks of 10k records and importing one by one.

mongoimport

As you can see in the above screenshot, the csv file column name became property name in BASON document. So each record of CSV file will insert as a new BASON document is specified collection.

Limitations

1) In few occasions error messages while importing is not detailed enough to diagnosis.
2) This tool do not support importing to collection that consists of embedded documents, if you do so it will simply ignore the embedded document part and import rest of the documents.
3) Can only import into a single collection, at a time.

Whatever the limitations are, for the very basic importing need, this tool is time saver.

Advertisements

One thought on “MongoDB Bulk Data Import From CSV

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s