Tuesday, 25 March 2014

PostgreSQL - pg_dump to copy PostgreSQL Database From One Remote Server to Another

While involved in Python + Django based projects and using postgreSQL as a database backend, there are occasions where I need to switch between servers and at times need to copy databases as well. For transferring the django application data + schema from one remote server to another, I use following ways as per convenience, available options, server restrictions etc.

1) Copying Django application data + schema (PostgreSQL) from one remote server by dumping the data in a file and then transferring that file to another remote server and restoring it. In this approach, you have got three options:

  • Dump both schema as well as Data to the file:
pg_dump -C -h host -U username db_name > /any_directory/dump_schema_and_data_file 
  • Dump only schema from PostgreSQL Database to a file:
pg_dump -Cs -h host -U username db_name > /any_directory/dump_schema_file
  • Dump only data from PostgreSQL Database to file:
pg_dump -a -h host -U username db_name > /any_directory/dump_data_file

After copying the above file to target server, you can easily restore database from file by using following command:
psql -h host -U username db_name < data_schema_file


2) Second approach I use to copy PostgreSQL database from one remote server to another is by directly using following command i.e without copying the data to intermediate file:

pg_dump -C -h localhost -U localuser db_name | psql remotehost -U remoteuser db_name

Thursday, 4 July 2013

Django - Install Psycopg2 on Mac OS X - pg_config path issue AND lipo: can't open input file - Issues Resolved

In one of my recent Django based projects, I happened to use PostgreSQL as Database Backend. In order to connect the db and for other operations with PostgreSQL, python's psycopg2 module is pretty popular and readily used. Psycopg2 got easily installed on the Linux (Ubuntu) machine but I faced some issues when I tried to install it on my Mac OS X 10.6.7. After following several tips and reading several blog posts, I finally managed to find the actual working solution which resolved the issues of installing Psycopg2 on Mac OS X. I also noticed that many people were trying to find solution of the same issues which I faced. So I am writing their solutions in a single blog post in a descriptive and step-by-step manner. There are basically two most common issues which I also faced while installing psycopg2 on Mac OS X:


PROBLEM 1:

When I tried to install psycopg2 on Mac OS X and entered following command on my terminal:

>pip install psycopg2

I got an error that pg_config file's path is not specified. Following is the full trace:

TRACE:
Error: pg_config executable not found.



Please add the directory containing pg_config to the PATH

or specify the full executable path with the option:



    python setup.py build_ext --pg-config /path/to/pg_config build ...



or with the pg_config option in 'setup.cfg'.

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build/psycopg2



SOLUTION OF PROBLEM 1:

Provided that some version of PostgreSQL is already installed, you will have to explicitly add the PATH of pg_config file in your PostgreSQL bin folder. In my case, the installation PATH for PostgreSQL is:

 /opt/local/lib/postgresql91/


so to define the PATH of pg_config file, I entered following command in my terminal:
> PATH=$PATH:/opt/local/lib/postgresql91/bin/


After running this command, the pg_config path error while installing psycopg2 was resolved


PROBLEM 2:

While installing psycopg2 on my Mac OS X, the pip install psycopg2 command returned following error:

lipo: can't open input file: /var/tmp//ccTy5xAu.out (No such file or directory)


Full trace of the error is as follows:

psycopg/psycopgmodule.c:951: fatal error: error writing to -: Broken pipe

compilation terminated.

lipo: can't open input file: /var/tmp//ccTy5xAu.out (No such file or directory)

error: command 'gcc-4.2' failed with exit status 1



SOLUTION OF PROBLEM 2:

After a googling a bit, and trying several recipes, the solution that worked for me was the following command which overrides the ARCHFLAGS architecture settings:

sudo env ARCHFLAGS="-arch i386 -arch x86_64" pip install psycopg2


This command finally installed psycopg2 on my Mac OS X without any issues.

Monday, 4 February 2013

Django Deployment - Fabric, GUnicorn, NGInx, Supervisor - Django-fagungis

Django has become a very popular web framework and I see lots of people confused about deploying Django projects the right way using the right tools. I myself faced a lot of problems in finding the right combination of tools for successful (and hassle free) Django Deployment.

Previously, I used Apache and mod_wsgi for Django Deployment but it required a lots of tweaking and found it to be a very lengthy process. You may go through this post to read whole account on Django Deployment issues using Apache and mod_wsgi.

After several cups of coffee and reading a lots of blogs, I found some alternatives/new-ways for Django Deployment. Those new ways/tools involved Nginx, gunicorn and supervisord with their lots of configurations. Still, when I started to use them, I found them more friendly for Django Deployment. But still I had to make some project related adjustments every time I used to deploy a new production-ready version of my project. Then I found this great tool "django-fagungis" (git - bitbucket) which promises a three step hassle free Django Deployment. After configuring it once, I just have to literally type three commands and its does my Django Deployment everytime like Aladdin's genie would have done for Aladdin. From cloning my project directly from bitbucket to configuring Nginx, Gunicorn and Supervisord on my remote server, it handles it all like a charm. You can find a detailed tutorial on how to use django-fagungis on its official site.

So, after following the initial configuration instructions of django-fagungis, I now have to type just three commands from my local machine!

Step1:
fab my_project test_configuration

Step2:
fab my_project setup

Step3:
fab my_project deploy

Sunday, 12 February 2012

Django Deployment - Setup mod_wsgi on CentOS

Following terminal command would install mod_wsgi on CentOS:

sudo yum install mod_wsgi
yum list installed mod_wsgi