Python Ecosystem - An Introduction


When developers shift from PHP, Ruby or any other platform to Python, the very first road block they face (most often) is a lack of an overall understanding of the Python ecosystem. Developers often yearn for a tutorial or resource that explains how to accomplish most tasks in a more or less standard way.

What follows is an extract from the internal wiki at my workplace, which documents the basics of the Python ecosystem for web application development for our interns, trainees and experienced developers who shift to Python from other platforms.

Install ImageMagick with PHP imagick extension on CentOS-5.8

You can easily install ImageMagick and PHP imagick extension from remi repository. There are some dependencies from epel repository too.

yum --enablerepo=remi,epel install ImageMagick.x86_64 php-pecl-imagick.x86_64

Wireless Ad-Hoc connection sharing in Ubuntu 10.10

  • dnsmasq-base has to be installed:

    sudo apt-get install dnsmasq-base

  • Remove dnsmasq because it conflicts with NetworkManager:

    sudo apt-get remove dnsmasq

  • Restart NetworkManager:

    sudo /etc/init.d/network-manager restart

  • Add a new wireless network with NetworkManager (left-click on NetworkManager icon, then select "Create New Wireless Network").
  • Set encryption to WEP 40/128-bit Key with a 13 characters key. (Note: You may have to experiment here according to what type of encryption with ad-hoc the device supports. WPA is not supported).
  • NetworkManager now should connect to itself (which means it creates the ad-hoc wireless network and routes any Internet traffic to your wired network interface). Now, connect with the client(s), and you should have a working Internet connection.
  • You have to make sure that this connection is shared to other computers and devices (clients). Make sure that the Connect automatically check-box is selected and on the IPv4 Settings tab make sure that the Method option is set to "Shared to other computers".

Updating OpenVZ vzctl on CentOS-5.8

While updating vzctl to latest on CentOS-5.8, I was getting the below error:

# yum update vzctl
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* openvz-kernel-rhel5:
* openvz-utils:
Excluding Packages in global exclude list
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package vzctl.x86_64 0:4.1-1 set to be updated
--> Processing Dependency: vzctl-core = 4.1-1 for package: vzctl
--> Processing Dependency: for package: vzctl
--> Processing Dependency: for package: vzctl
--> Running transaction check
---> Package libcgroup.x86_64 0:0.37-4 set to be updated
---> Package vzctl-core.x86_64 0:4.1-1 set to be updated
--> Processing Conflict: vzctl conflicts ploop-lib < 1.5-1
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package ploop-lib.x86_64 0:1.5-1 set to be updated
--> Processing Conflict: ploop-lib conflicts vzkernel < 2.6.32-042stab061.1
--> Processing Conflict: ploop-lib conflicts vzkernel < 2.6.32-042stab061.1
--> Processing Conflict: ploop-lib conflicts vzkernel < 2.6.32-042stab061.1
--> Processing Conflict: ploop-lib conflicts vzkernel < 2.6.32-042stab061.1
--> Finished Dependency Resolution
ploop-lib-1.5-1.x86_64 from openvz-utils has depsolving problems
  --> ploop-lib conflicts with ovzkernel
Error: ploop-lib conflicts with ovzkernel
You could try using --skip-broken to work around the problem
You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.

Turns out that ploop is no longer required for vzctl on CentOS-5.8 and can be removed:

yum update problem on CentOS 5.8 server

"Since you have RHEL5-based kernel that do not require ploop, you can remove ploop when installing vzctl-4.0. I have made vzctl not requiring ploop by dynamically loading it when it's available. Note that vzctl is not requiring ploop anymore, it just conflicts with the old version of it."

The solution was to remove ploop in single transaction as mentioned:

# yum shell
> update vzctl
> remove ploop\*
> run
> quit

mysqlbinlog Tips and Tricks


So, you have a binlog. You want to find out something specific that happened inside of it. What to do? mysqlbinlog has some neat features, which I thought we would look at here.

I should first explain what mysqlbinlog really is. It is a tool that lets you analyze and view the binlogs/relaylogs from mysql, which are stored in binary format. This tool converts them to plaintext, so that they’re human-readable.

iscsi notes

iscsi target

  • Install iscsi utils:
    yum install scsi-target-utils
  • Bring up tgtd:
    chkconfig tgtd on
    service tgtd start
  • Define an iscsi target name:
    # tgtadm --lld iscsi --op new --mode target --tid 1 -T
  • Delete specific iscsi target:
    # tgtadm --lld iscsi --op delete --mode target --tid 1
  • Add a logical unit to the target:
    # tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /dev/vg1/lv1_iscsi0
  • Delete logical unit:
    # tgtadm --lld iscsi --op delete --mode logicalunit --tid 1 --lun 1
  • Enable the target to accept any initiators:
    # tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
  • View configuration:
    # tgtadm --lld iscsi --op show --mode target
  • Save the tgt conf:
    # tgt-admin --dump >/etc/tgt/targets.conf

iscsi initiator

  • install the initiator client:
    # yum install iscsi-initiator-utils
  • startup the iscsi daemon so all kernel modules get loaded:
    # chkconfig iscsid on
    # chkconfig iscsi on
    # service iscsid start
  • Discover targets:
    # iscsiadm --mode discovery --type sendtargets --portal
  • Login to target:
    # service iscsi restart
  • To manually login:
    # iscsiadm --mode node --targetname --portal --login
  • To manually logout:
    # iscsiadm --mode node --targetname --portal --logout
  • To delete targets:
    # iscsiadm --mode node --targetname --portal -o delete
  • List sessions:
    # iscsiadm -m session
  • List nodes:
    # iscsiadm -m node

Accessing ssh servers behind NAT

Create a "config" file in your "~/.ssh" directory with the below contents:

Host server1
  HostKeyAlias server1
  CheckHostIP no
  Port 221

Host server2
  HostKeyAlias server2
  CheckHostIP no
  Port 222

The key is to set CheckHostIP to "no" and use "HostKeyAlias" to specify an alias that should be used instead of the real host name when looking up or saving the host key in the host key database files.

The Port line avoids having to specify the port when connectig.

Connect to corresponding host via:

$ ssh {user}@server1
$ ssh {user}@server2

strace all apache child processes

# ps h --ppid `cat /var/run/` -o pid | awk '{print "-v -ff -tt -T -s 1024 -o /tmp/strace.out -p " $1}' | xargs strace

This will attach strace to each of the apache child processes.

-v verbose
-ff with -o will log the output to "/tmp/strace.out.{pid}" and follow forks.
-tt prints the timestamp of each call.
-T prints the duration of each call.
-s specifies the maximum size of the output string to more than the default 32.

Related links:

NFS for clusters


Here are notes I've found useful for configuring reliable shared disk on a linux cluster...

md5 and sha1 digest with openssl

md5 digest:

echo -n 'md5 digest of text' | openssl dgst -md5

sha1 digest:

echo -n 'sha1 digest of text' | openssl dgst -sha1

Syndicate content