Please login or register to participate.
Discussion
.
offray Mar 14, 2010 03:39 PM
I want to understand backups better in this scenario: I have two Zope instances in two different severs with two plone/cynin instances for different sites/communities (they even run different versions of cyn.in). I want to move it to a single Zope instance with two cyn.in instances. As far as I have understand, you just need to backup Data.fs to have all your zope instance backup, but I don't know how to integrate the two Data.fs files of the different Zope instances so, they can be part of a single one, but keeping separate data for every cyn.in instances. Made I myself clear?
Replies (8)
offray Mar 14, 2010 04:21 PM
Looking for the answer to myself I can see that when I run the Zope option for backup, going to /manage then to Control Panel > Database > Main and finally Packing it. Is supposed this will give me a Data.fs.old, but looking for Data* I see:

./3.1/var/filestorage/Data.fs.tmp
./3.1/var/filestorage/Data.fs.index
./3.1/var/filestorage/Data.fs.lock
./3.1/var/filestorage/Data.fs

Some of them should not be Data.fs.old?

Another thing is that looking in /bin for scripts to make backups I see:

backup repozo restore snapshotbackup snapshotrestore

Can you tell me the differences between them, and if one of them (or several) are supposed to be ran to backup and if this can be done while the cyn.in instance is still running?
dhiraj Mar 17, 2010 10:13 PM
Running a single Zope instance with two separate file source code bases is not possible, really. As far as I see, you'll only get into trouble - you need 2 separate source code bases if you want to run 2 sites with different Cyn.in versions.

./bin/backup - This does an incremental backup. Backups are stored at <buildoutfolder>/var/backups
./bin/snapshotbackup - This does a full backup, your entire Data.fs will be backed up at <buildoutfolder>/var/snapshotbackups
./bin/snapshotrestore - This should really only be attempted if you are running on the same actual source code base version, different versions may cause unpredictable results.

A better idea to take a snapshotbackup across different versions is to rename the backup file to whatever.gz, then gunzip it and rename the single file outputted to Data.fs in your new source code base and then doing a reinstall of ubify.policy. Re-installing ubify.policy is *heavily* recommended when you move a Data.fs file across version trees - unless you really know what you're doing.
offray Mar 19, 2010 04:22 AM
Thanks for your quick answer as always. I made not myself clear. This is what I have now:

Server 1 -> Zope 1 -> Cyn.in 3.04 powered Site -> Community 1 Contents
Server 2 -> Zope 2 -> Cyn.in 3.1 -> Community 2 Contents

And this is what I want to have:

                                 ----> Cynin 3.1 powered site -> Community 1 Contents
Server 2 -> Zope 2 -|
                                 ---- > Cynin 3.1 powered site -> Community 2 Contents

So I want to migrate community one contents so they can run on the same zope that is powering community 2 contents and also with the same version of cyn.in. Is this possible? I imagine so. But if this is the case. When I made a backup the Data.fs will contain the contents of the two communities or there is a way to split them so one backup doesn't mix the data of the other.
dhiraj Mar 19, 2010 10:19 AM
Ah, so you want to do what we call a "shared multi-site single instance" scenario.

Hmm... the approach is problematic, at best.

Try having a look at this product: http://pypi.python.org/[…]/0.5

You'll be able to create 2 separate file storages for your 2 Data.fs files and run them on a single instance, with some research and tweaking.

Warning #1: *First* try this on a test system, please!
Warning #2: There are some inherent problems with doing this - I would recommend that you run multiple instances, 1 per site, that's the maximum flexibility + power you'll get.
offray Mar 19, 2010 07:22 PM
I think I'm almost there trying to made myself clear. One Zope Instance can support multiple plone sites. And every plone site can have one cyn.in instance Isn't it?. This would be the picture:

                          -----> Plone Site 1 --> Cynin (3.1) Site 1-> Community 1 Contents
Server -> Zope -|
                          -----> Plone Site 2 --> Cynin (3.1) Site 2-> Community 2 Contents

your advice is that is better to have something like:

                    ---> Zope 1 --> Plone Site 1 --> Cynin (3.1) Site 1-> Community 1 Contents
Server -> -|
                   ----> Zope 2 --> Plone Site 2 --> Cynin (3.1) Site 2-> Community 2 Contents

If that is the case the idea is to copy all contents of the cynin build in a folder and run "/instance startup" in each folder, so what I need is to change the port where one instance is running (say 8081) and have the other one on default port (8080). I was thinking in the first approach because I thougth that this will save resources if I run just one Zope. I will test the second approach, but I'm still curious about the second one and the saving of resources (is this notable? or is almost the same as running two Zope servers).
dhiraj Mar 24, 2010 01:55 PM
Yes, you got the architecture right.
To change the port you have to modify user.cfg in your copied location and then do buildout.

Resource savings will actually be probably ... lesser with this way, actually. It will be exactly like running 2 zope servers because that's what you'll be technically doing, on one machine. But my point was not really about savings, but about future flexibility requirements, etc.

Suppose you *do* manage to get 2 sites running shared mode on 1 zope server, and take it into production usage, then *later* when you want to have 1 of the sites run separately on another sever you would again have to figure out how to split them apart, etc.
offray Mar 24, 2010 02:45 PM
Ok. Thanks a lot. I will be testing this escenario this week and keep you posted :). By the way I have seen that Data.fs in our community environment is huge (15GB) but surely we don't need all this information. We would like to make some cleaning and reduce the size of the images. Is that possible? How can I see which are the biggest contents/spaces in our community instance to program the clean up. Also, it's possible to set up a maximum size for content tipe, say images should be maximum of 2 mb, videos 100 mb and so on?
dhiraj Mar 24, 2010 02:57 PM
You have probably accumulated a lot of undo data in your Data.fs that will go away when you "pack" the ZODB. You can find this in http://<CYNINSITE>:8080/manage and then click through Control panel -> Database Management -> main. Change the value in the textbox to 1 days and then hit the Pack button.

This will take *really* long, especially on a server with slow I/O or with lots of data like in your case.
 
Loading