I wish i could say i have backups finished done. Unfortunately my computers are just as disorganised as me and so i’m not certain yet without checking a few things which is the latest version i have. I’ll check with the repository but i want to take an abstract look at the whole thing and see if i can work out what needs adding, what doesn’t need adding to get something good enough to release before testing it and sending it out there.

I’ve had quite a busy week as i had to go to uni on monday and then had a job interview tuesday and its the last week before people go off to uni so a last chance to see some friends for a while. I’m still no further forward with my test machine’s problems. It seems ubuntu is not suitable for the job at all. I’m probably going to decide on and buy a monitor for the machine in the next few days and it will become my main box, probably likely running slackware because slackware is not plauged by broken software. I cannot believe that a modern linux distribution would have trouble staying awake on an AMD 64 X2 that is brand new.

i’m not sure why but currently the machine i use to work on webcp keeps cutting out over the network…. it’s not an overheating problem and its happening with either of the two 3com 3c905 10/100Mbps network cards in there….. i can’t check it at the moment because it hasn’t got it’s own monitor (been meaning to buy one for a while). Its running ubuntu and shows no signs of whats up.. i’ll boot it up and check the sys log to see if there’s anything helpful.. Maybe the network cards are to blame because i’ve seen a freeze before with completely different hardware in the machine except those cards. It has onboard LAN as a backup anyway.

On another note I changed the backups code recently (just after i got it working pretty well) to use arrays to keep track of what backup archives are being dealt with, its not working well though. I was attempting to solve the problem of how to move the data about backups around between webcp hooks but i think my only option is to serialize individual parts.
EdG

Today, I’ve committed a new module to fully support the Zimbra mail suite from within web-cp. If you aren’t familiar with Zimbra, it is a feature-complete open source replacement for Exchange Server. It provides full mail, calendar and document sharing functionality (integrated) as well as mobile connectivity and Outlook support (as commercial add-ons). The webmail is AJAX based and feels like a desktop mail application. Also integrated are spam and antivirus scanning. The new module utilizes the Zimbra administrative soap interface with transport via SSL to ensure the security of all management functions.

I’ve been a little busy this week and also the PC that i use to test web-cp is waiting for some memory at the moment.. when that arrives i can use it again.. in the meantime i still have the latest source on my laptop so i can work on that outside. If that PC becomes my desktop then i will have a lot more power to work with and virtual machines will be a lot easier to work with. The issue of recursive backups is coming together nicely though. As i said its been a busy week due to recent birthday and looking for a weekend job. Now i have more time on my hands
EdG

This is what a backup of domain.com currently contains:

domain.com_web.tar
domain.com_webcp_data.txt
domain.com_metadata.txt
ironman_1185832487.tar

In order, domain.com_web.tar is the backup from the apache module for domain.com, domain.com_webcp_data.txt is the backup of the info webcp holds about the domain, domain.com_metadata.txt is the metadata file containing a list of files in the backup. ironman_xxx.tar is a backup of the ironman user from the domain.

OK there need to be some changes, probably domain.com_web.tar should become domain.com_apache.tar, ironman_xxx should become domain_ironman_xxx (because unix_users are different in 0.6). It is a start though. I still have to refine things and this is only domain backup but that means everything else should work ok without much modification. At the moment backup file names just come from the module but you could easily get the module to call a hook from the backups module to get the name (something Gyrbo suggested and i think pdrake suggested as well).

Whether the tar files need times in their names or indexes or seals or anything like that i don’t know. Also currently data is only serialized between the frontend and backend parts so nothing is serialized in the backup_manager itself…. it could easily be done if necessary for security though. Also for security and performance forking might be a good idea for each backup to be forked by the backup manager hook. Forking of hooks that the backend calls for a module may be a good feature and would be one way of ensuring backups are forked.

I think webcp could benefit from a method of exchanging data between different hooks rather than just being able to use data between functions in the same hook. I would hope this could be done with variables rather than file or database access.
EdG

ps any suggestions are welcome and as i said there’s quite a bit still to be done but i’ve finally got a system that can make a complete backup of a domain and all of its selected users.

Right now the main thing i’m working on is how a backup of a reseller/domain is done recursing through the users and domains to be backed up. At the moment a reseller backup will provide a choice of domains to backup but all users under those domains will be backed up so backing up the domains and users is part of this. Also how various hooks work together to build the backup. A backup manager function is in charge which calls the relevant hooks.

I’m beginning to realise how useful some of the techniques of last years computer science lectures are in this. E.g. splitting things into functions where code gets too complex and the manjana(sp?) principles like if you need a specific function to do something write the empty function and write the rest as if that function exists, then work on the function, allowing you to skip thinking about the specifics for the moment…

Also i can’t imagine my life without a text editor like Crimson editor it is without a doubt the best and most functional editor i’ve ever found. Most useful are things like opening and saving files to a remote ftp server so i can save direct to my test machine and the ability to have multiple files open in tabs, also indenting, saving a backup file every time you save and conversion to unix file types are great. It has a huge number of features and i would recommend it to anyone.

I found the project features useful when manually editing Visual Basic form files for my a level computing project

For anyone not developing on windows i would recommend something called CUTE which is an awesome text editor….although you’ll probaly have to compile it its worth the trouble. Or there’s always GEdit.
EdG

I was just thinking about how every user of webcp is a user at some level but if you want to backup domains you have to look at domains, not the users who own them, makes sense but would it be easier if we just used various users inside webcp and store domains as users? obviously there has to be a concept of a domain just would it be easier if a user was more than the user concept we have now.

Okay, i thought i’d provide an update as to what’s going on with the backups module. Everything functions fine as to creating a backup and the process for restore is very similar but putting domain backups inside reseller backups and things like that are more complicated. Due to this i have followed the policy of one run of backup.php means one reseller, domain or user backup so far.

I have since decided to move backup.php to backups_backup and to make this run as a hook. The alternative was forking a backup function (which would have been very similar to just calling the hook anyway). If the hooks method doesn’t work out i will go back to the forking idea. But for now i’m working towards easy maintainability which means hooks.

I just committed an update in the CVS to webcp.php to improve the method used to tell the backend to call a hook. This now allows you to send $data to it in the $extra part of commit with the module to run rather than just the module to run. Whether data is global or not isn’t really helpful to me because obviously the backend and frontend are seperate so data has to be sent anyway. Now you can call a hook with some data in the backend from the frontend. In future i think forking the call_hook would be a good idea for security and performance. Maybe the seal should be checked or something i don’t know.

The idea is to register hooks as follows to allow a single call to call_hook(’server:commit:backup’, $data) to run the whole process from start to finish.

// Setup backup hooks
register_hook('server:commit:backup', 'backups_backup');
register_hook('server:commit:backup', 'backups_generate_archive');
register_hook('server:commit:backup', 'backups_ftp_archive');
register_hook('server:commit:restore', 'backups_ftp_archive');
register_hook('server:commit:restore', 'backups_restore');


Clearly calling server:commit:backup will be done by webcp.php using commit(’module’….). The idea is that the backup, generate_archive, ftp_archive and any other functions will all run on the data in sequence (i may have to sepcify priority to confirm they run in order i know there’s a way to do that with register_hook). This should make maintaining the module simple if all goes to plan.

Right now i’m working on moving to hooks rather than relying on calling backup.php from the backend with specific data and gathering it later.
EdG

Lately, it’s been quite easy to spot that I’m starting to dislike PHP. Over the years I’ve worked with several different languages and all of them have their downsides. These usually pop up when you’re doing some advanced stuff or doing things the language wasn’t made for. It may be that these little annoyances are becoming evident because I’ve been using PHP the longest. Here’s good summary problems I ran into while trying to update the web-cp modules/hooks system: http://keithdevens.com/weblog/archive/2003/Aug/13/HATE-PHP
His last point has since been resolved, though.
Other little things that both me are the inconsistencies in the standard library: http://tnx.nl/php (just ignore the pro-Perl talk).
In addition, there’s the extensibility (or better, lack thereof). Good steps have been taken with PEAR, but it still isn’t (IMHO) up to par with other languages like Perl (CPAN), Java (myriad of extensions, etc.), Python (extensive standard library and plenty of extensions), …
My main gripe is that you can never be sure the user has an extension that you require. With Web-cp, we are in a privileged situation since it’s easy to add certain extension to the requirements. Few projects have that luxury. This is of course not about PHP-only extension (such as those in PEAR). The problem with those is that they’re often quite slow.
Finally, there’s that little voice in the back of my head that’s worried about security. The PHP developers have show a distinct lack of interest in pretty much anything security related: http://www.php-security.org/ Some of those bugs are dangerously old. This should be a major concern to us as web-cp developers since we’re running PHP as root!

To finish off, I don’t believe PHP is completely useless and I certainly won’t stop working on web-cp because of it. It’s just that if I would start over, I would probably use a different language. Which one that would be is still in the middle, though.

My next blog post will most likely be about how I envision web-cp in the (far) future. Things like Object-Relational Mapping and a data-driven interface come to mind. And yes, it will probably still be in PHP .