Please login or register to participate.
Wiki Page

Quickstart buildout cyn.in 3.2.2 on Ubuntu 12.04 LTS 64bit

.

Setup cyn.in 3.2.2 on Ubuntu 12.04 LTS 64bit - Step by Step Instruction

 

I have setup an productive server based on OpenSource version 3.2.2 of cyn.in based on actual supported Ubuntu 12.04 LTS. I am shure, that this description and the steps inside need some improvements. But overall it is runnig at the end. Feel free to comment, if you have suggestion for improvement or newer versions of Plone modules and add ons. 
In a separate wiki I will describe how to setup a scalable zeo cluster based on this installation with apache2 web server, ssl and apache2 load balancer. 
To understand, you need knowledge about Linux administration und you must know the background of buildouts with Zope/Python and Plone.

 

Installation Ubuntu 12.04 LTS (12.04.3) 64bit

Information about: http://wiki.ubuntuusers.de/Precise_Pangolin
ISO-Download: http://releases.ubuntu.com/precise/ubuntu-12.04.3-server-amd64.iso
Search for installation instructions depending on environment: https://www.google.com/search?q=ubuntu+iso+installation

Server preparation

If server system is up and running, check intent access is working. so login and 

$ ping 74.125.136.139

and check DNS configuration

$ ping google.com
If this fails, please fix before continue. 

Now change user to root:

$ sudo su

Install all updates available:

$ apt-get update
$ apt-get upgrade

Clean up installation environment

$ apt-get autoclean
$ apt-get autoremove

Install monitoring tool:

$ apt-get install htop 
More about htop: http://www.google.com/search?q=install+htop)

Create new user cynin with group cynin. Add user cynin to group sudo

$ adduser cynin

Add user cynin to group sudo

$ adduser cynin sudo

Add a lot of missing packages within ubuntu 12.04 for cyn.in

$ sudo apt-get install build-essential libssl-dev libjpeg62-dev libreadline-gplv2-dev wv libxml2-dev libxslt1-dev libsasl2-dev poppler-utils libdb-dev libldap2-dev 

Build custom python2.4

Build python2.4 from source, like this (according to: http://ubuntuforums.org/showthread.php?t=1976837) as user cynin:

$ cd ~
$ sudo apt-get build-dep python
$ wget http://www.python.org/ftp/python/2.4.6/Python-2.4.6.tar.bz2
$ tar xvjf Python-2.4.6.tar.bz2
$cd Python-2.4.6

Create a patch file setup_py.patch:

--- setup.py    2006-10-08 19:41:25.000000000 +0200
+++ setup.py        2012-05-08 14:02:14.325174357 +0200
@@ -269,6 +269,7 @@
         lib_dirs = self.compiler.library_dirs + [
             '/lib64', '/usr/lib64',
             '/lib', '/usr/lib',
+           '/usr/lib/x86_64-linux-gnu'
             ]
         inc_dirs = self.compiler.include_dirs + ['/usr/include']
         exts = []
@@ -496,7 +497,8 @@
                 ssl_incs += krb5_h
         ssl_libs = find_library_file(self.compiler, 'ssl',lib_dirs,
                                      ['/usr/local/ssl/lib',
-                                      '/usr/contrib/ssl/lib/'
+                                      '/usr/contrib/ssl/lib/',
+                                     'x86_64-linux-gnu'
                                      ] )
         if (ssl_incs is not None and

and apply the patch:

$ patch setup.py < setup_py.patch

Activate ssl-support:

$ nano ./Modules/Setup.dist
# CSV file helper
#_csv _csv.c
# Socket module helper for socket(2)
_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/usr/lib/ssl
_ssl _ssl.c \
   -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
   -L$(SSL)/lib -lssl -lcrypto

# The crypt module is now disabled by default because it breaks builds

Now let's compile:

$ env CPPFLAGS="-I/usr/lib/x86_64-linux-gnu" LDFLAGS="-L/usr/include/x86_64-linux-gnu"  ./configure --with-threads --enable-unicode=ucs4 --with-readline
$ make

Check for major problems

$ make test

if non, install:

$ sudo make install

Check python:

$ python2.4 -V

with output: python 2.4.6
and SSL functionality

$ python2.4 /usr/local/lib/python2.4/test/test_socket_ssl.py

with output:
test_rude_shutdown ...
test_basic ...
test_timeout ...

Installation of cyn.in 3.2.2

After a cup of coffee, let's start with installation of cyn.in 3.2.2:

$ cd ~/src
$ wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz
$ tar xzf setuptools-0.6c11.tar.gz
$ cd setuptools-0.6c11
$ sudo python2.4 setup.py install 
 $ sudo easy_install-2.4 -U virtualenv==1.7.1
$  virtualenv --python=python2.4 ~/venv
$  ~/venv/bin/easy_install-2.4 ZopeSkel
$ ~/venv/bin/easy_install-2.4 lxml==3.3.6
$  ~/venv/bin/easy_install-2.4 python-ldap
$  ~/venv/bin/easy_install-2.4 http://dist.plone.org/thirdparty/PIL-1.1.7.tar.gz
$ ~/venv/bin/easy_install-2.4 ftp://xmlsoft.org/libxml2/python/libxml2-python-2.4.30.tar.gz
$ sudo apt-get install subversion
$ svn co http://odn.cynapse.com/svn/cynin/tags/cynin_3_2_2/

Now we have lastet source of cyn.in and a python environmet for first buildout. 
Let's make some missing directories

$ cd cynin_3_2_2
$ mkdir eggs
$ mkdir ./var
$ mkdir ./var/filestorage

bootstrap.py from cyn.in subversion will not work with setuptool, so take onother one

$ rm bootstrap.py
$ wget http://downloads.buildout.org/1/bootstrap.py
$ sudo ~/venv/bin/python2.4 bootstrap.py -v 1.7.1

Take care for versions of Plone modules

Now we have to pin/change several versions in directory ~/cynin_3_2_2/buildout-configs/:
Any hints for newer versions are very much appreciated!
$ cd ~/cynin_3_2_2/buildout-configs

Pin version of precompiler:

Add to ploneversions.335.cfg

[versions]
# Buildout infrastructure
plone.recipe.precompiler = 0.3

In same file

change PIL entry from 1.1.6 to version 1.1.7

# External dependencies
PIL = 1.1.7

Add to section

# Plone releases
Products.LDAPUserFolder = 2.20

Add at the end of file

dataflake.ldapconnection = 1.2
dataflake.cache = 1.2

and change in ./buildout-configs/pil.cfg:

eggs = PIL
$ cd ~/cynin_3_2_2

Install additonal and newer products and Hotfixes

Install available Plone-Hotfixes and other stuff to  customproducts.cfg in ./buildout-configs/

[custom]
eggs =
        Products.Zope_Hotfix_20110622
        Products.PloneHotfix20121106
        Products.PloneHotfix20130618
        Products.PloneHotfix20110720
        Products.PloneSlimbox
        Products.contentmigration==2.0
        Products.CMFNotification==2.2
        Products.OpenXml==1.1.1
        Products.Scrawl
        Products.PloneSlimbox==0.7.1
  
zcml =
        Products.contentmigration
        Products.CMFNotification
        Products.OpenXml
        Products.PloneSlimbox
        Products.Scrawl

Delete some old products now integrated as eggs from ~/cynin_3_2_2/products

$ sudo rm -r ~/cynin_3_2_2/products/PloneHotfix20110720
$ sudo rm -r ~/cynin_3_2_2/products/PloneSlimbox/
$ sudo rm -r ~/cynin_3_2_2/products/CMFNotification/
$ sudo rm -r ~/cynin_3_2_2/products/OpenXml/
$ sudo rm -r ~/cynin_3_2_2/products/Scrawl/

Ready to rumble:

./bin/buildout

Much luck and have fun!

Setup productive environment:

$ cd /opt
$ sudo mkdir cynin
$ cd cynin
$ sudo mkdir buildout
$ cd buildout
$ sudo cp -R /home/cynin/cynin_3_2_2/* .
$ sudo chown -R cynin:cynin /opt/cynin
$ python2.4 bootstrap.py -v 1.7.1
$ easy_install -U Distribute
First step of tuning: Add to file ./buildout-configs/plone3.instance.cfg  in paragraph 
[instance]
zodb-cache-size = 50000
zeo-client-cache-size = 200MB

Now ready for last step to heaven

$ ./bin/buildout

Yeah! If everything went right, you can start cyn.in sever by

$ ./bin/instance start
Check Server using Browser-URL for Zope backend[server-IP]:8080/manage
User: admin
Password: secret
Congratulations! 

Apache webserver already installed? 

$ sudo apt-get install apache2
You may take apache2 configuration file cynin.conf delivered by cyn.in ISO or take this one. Put it into /etc/apache2/sites-enabled

If you want to start cyn.in durign server boot, you can take the script /etc/init.d/cynin from ISO or take this one. Register script for autostart:

$ sudo update-rc.d cynin defaults
All done? Let's drink a beer.
Description
Yes, it's up and running!
Comments (31)
tobym Oct 24, 2014 04:21 PM
This looks great, just what I'm after, I have a problem though. This line:

$~/venv/bin/easy_install-2.4 lxml

doesn't work, I get this output:

Searching for lxml
Reading http://pypi.python.org/simple/lxml/
Best match: lxml 3.4.0
Downloading https://pypi.python.org/pac[…]c4e4ee04e1f8290ae0f70e34eea
Processing lxml-3.4.0.tar.gz
Running lxml-3.4.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-sMNlv7/lxml-3.4.0/egg-dist-tmp-7JLHn3
This lxml version requires Python 2.6, 2.7, 3.2 or later.
error: Setup script exited with 1

The Python version you install seems quite specific, so I'm loathe to mess around with that - can you recommend a way of rectifying this?

Thanks,

Toby
tobym Oct 24, 2014 04:45 PM
I managed to get past this issue by doing this:

$~/venv/bin/easy_install-2.4 lxml==3.3.6

Not sure if that's a good thing to do? I don't know whether this ultimately will work, as the Cyn.in subversion seems to be inaccessible, hopefully only temporarily?!
gsiak Nov 13, 2014 03:51 PM
changelog of lxml 3.4.0 (http://lxml.de/3.4/changes-3.4.0.html) says:

"Requires Python 2.6, 2.7, 3.2 or later. No longer supports Python 2.4, 2.5 and 3.1, use lxml 3.3.x for those."

So 3.3.6 will be fine!
tobym Nov 13, 2014 03:56 PM
Hey, thanks for getting back to me!

I actually ran into a more fundamental problem, in that the subversion repo you used no longer seems to be available (and the github version of the code seems like it might be quite different from what you used) - is there still a way of getting (something like) your method to work?

I'm also struggling with the 'official' buildout (http://cyn.in/home/cyn.in-developers/quickstart-buildout), I am having a fair bit of trouble with that too.

I'd really like to be able to do a build to a more modern OS than Debian Lenny (which is what the compiled open source version comes with)!
gsiak Nov 13, 2014 04:08 PM
No trouble, cynpase moved from ODN to GITHUB. You can use 3.1.3 too. Difference is the structure of .cfg-files in a subdirectory in 3.2.2 instead having all configuration information in buildout.cfg.

https://github.com/cynapse/cynin
tobym Nov 13, 2014 04:10 PM
Thanks - that's what I've been trying to use with the official buildout. I'll try using it with your guide
gsiak Nov 13, 2014 04:11 PM
PLS give a feedback if this will be ok and I'll change the guide.
tobym Nov 14, 2014 12:48 PM
Hello again. I've done most of 'Take care of versions of Plone modules', I'm not sure about one step though. In case it's useful I'll tell you what I have done.

You say:
"Add to ploneversions.335.cfg

    [versions]
    # Buildout infrastructure
    plone.recipe.precompiler = 0.3"

I added this at the top of https://github.com/cynapse/cynin/blob/master/versions.cfg

You say:
"change PIL entry from 1.1.6 to version 1.1.7

    # External dependencies
    PIL = 1.1.7"

I did this here https://github.com/[…]/versions.cfg#L22

You say:
"Add to section

    # Plone releases
    Products.LDAPUserFolder = 2.20"

I did this in the same place as the previous one.

However, then you say:
"and change in ./buildout-configs/pil.cfg:

    eggs = PIL"

I don't know where to do this - if I grep "eggs =" I get lots of hits, including a few in https://github.com/cynapse/cynin/blob/master/buildout.cfg - I'm guessing I need to change something in this file, I don't know what though?

Any help would be very gratefully received!

Thanks,

Toby
tobym Nov 17, 2014 05:28 PM
If you do get a chance to look at this, my last comment was basically a long way of saying, "where should I set eggs = PIL"!
gsiak Nov 17, 2014 05:48 PM
This is the main difference between 3.1.3 and 3.2.2. All configurations are now in buildout.cfg. YOu are right to put it there, So this guide has to be changed in this lines.
tobym Nov 17, 2014 05:58 PM
Thanks for getting back to me! The thing is, I don't know where to make this change - I realise it may be hard for you to say, but I have no idea?! I wonder if it's somewhere in this file https://github.com/cynapse/cynin/blob/master/buildout.cfg - I'm just guessing though?!
gsiak Nov 17, 2014 06:14 PM
pil.cfg looks like this, which works fine for me. I'm sure you will find the right place with this information:

[buildout]
find-links +=
    http://dist.repoze.org/
eggs +=
        ${pil:eggs}
parts +=
        pil
[pil]
eggs =
        PIL
[pil]
recipe = zc.recipe.egg==${versions:zc.recipe.egg}
index = http://dist.repoze.org/legacy/simple/PIL/
eggs = PIL==1.1.7
tobym Nov 17, 2014 06:31 PM
Thanks again - do I need to *add* this line (eggs = PIL==1.1.7) somewhere?

In the *.cfg files in the github version of the project, there is no "[pil]" section at all, so I can't figure out where this would go.

Thanks for your help so far, I really appreciate it!
gsiak Nov 17, 2014 06:36 PM
IMHO you can put this into versions.cfg too.
tobym Nov 17, 2014 06:38 PM
Ok, great - all of this?

[pil]
recipe = zc.recipe.egg==${versions:zc.recipe.egg}
index = http://dist.repoze.org/legacy/simple/PIL/
eggs = PIL==1.1.7
gsiak Nov 17, 2014 06:43 PM
No, only the line PIL == 1.1.7 which will bound the version.
P.S.: PIL is only configured in 3.2.2, in 3.1.3 you can remove this whole staff.
tobym Nov 17, 2014 06:53 PM
Ok...so I'll just skip this part and try the buildout!
tobym Nov 18, 2014 11:54 AM
If you have a chance to look at this, I have tried the buildout and have more feedback:

1. I got an error that lxml 3.4.0 couldn't be installed, so in this file:

/eggs/openxmllib-1.0.7-py2.4.egg/EGG-INFO/requires.txt

I changed "lxml>=1.3.0" to "lxml<=3.3.6"

2. I got the error, "UnknownExtra: zope.i18n 0.0 has no such extra feature 'zcml'" - Google told me that *maybe* I should add this to buildout.cfg, after "fake-zope-eggs = true":

skip-fake-eggs =
      zope.i18n

3. Buildout now fails with, "Error: Couldn't install: zope.container 4.0.0" - and I'm stuck

You are probably very busy, but if you had any more tips I'd be very grateful!

Toby
tobym Nov 18, 2014 11:57 AM
PS The error detail was:

In file included from src/zope/container/_zope_container_contained.c:35:0:
include/persistent/cPersistence.h:19:25: fatal error: bytesobject.h: No such file or directory
compilation terminated.
gsiak Nov 18, 2014 12:27 PM
T had some trouble with lxml too. Please try 'plone.uuid = 1.0.0' in versions.cfg file after removing you changes.
tobym Nov 18, 2014 12:29 PM
Thanks! Which changes should I remove?
gsiak Nov 18, 2014 12:32 PM
You wrote: I changed "lxml>=1.3.0" to "lxml<=3.3.6"
tobym Nov 18, 2014 12:36 PM
That worked, I still have the compilation problem for zope.container though
tobym Nov 18, 2014 12:58 PM
Yes, I've been looking at that page, I can't find anything to help with the zope.container problem though - I'll keep looking
tobym Nov 18, 2014 01:07 PM
Apologies, I see you were pointing out the part about btrees failing. That looks promising, I don't actually understand the solution though, I can't figure out what needs to change?
tobym Nov 24, 2014 07:14 PM
Hello! I appreciate you're probably very busy, but if you ever have a chance to return to this let me know how you get on - I didn't get any further unfortunately, I couldn't see how to use the information in that (btree) link to make any more progress.
gsiak Nov 24, 2014 07:33 PM
As the headline says: The guide is for version 3.2.2, which ist no longer available. Second, some components, which are not fixed by version ID, are updated and will no longer work. I've tried a buildout with '-n'. Installation stopped working and I had to restore the olf one, because it was 'damaged' by components, which stopped supporting Python 2.4.
As the rsult, I have to start with a fresh installation and to write a new guide. This guide has to be checked regulary for incompatible new versions of alls components which are still not fixed.
BTW: I have successfully updated an cynin-installation with plone 3.3.6 coming from 3.3.5. 3.3.6 is a bugfix update for plone and the last version for 3.3.x.
tobym Nov 25, 2014 11:20 AM
Yes, I realise I've been trying to use your guide for a different version. If you do write a new guide I will look forward to trying it out! I would really like to be able buildout Cyn.in onto a more robust OS, I don't know enough about Plone etc etc at the moment though. If there was anything I could do to help, however, let me know.
 
Loading