Archive
You are currently browsing the cormander blog archives
for April, 2008.
By cormander
OK folks, I’ve revamped my test box’s mail system and everything is working like a charm. Here is the breakdown of how future versions of the 0.7.x series of ravencore will look:
postfix-2.5 built with the sqlite patch and dovecot sasl support
dovecot-1.0 with the sqlite extension
sqlite 3.5.x libraries
All of the above will be published in a yum repo, and will be marked as dependencies for the ravencore package. Just install this .repo file to your /etc/yum.repos.d/ directory and “yum install ravencore”, it’ll grab all of these for you and update them on the system. This will also make what distributions ravencore supports very clear; it will always be the 2 latest CentOS distributions, and the 2 latest Fedora distributions (CentOS 4 & 5, and Fedora 8 & 9)
Depending on how much help I get from other developers, I may or may not publish apt repos for debian packages. That is up in the air for the moment.
RavenCore will no longer use the one-file-per-line format for postfix and dovecot configuration files; it’ll instead use templates: main.cf.in, dovecot.conf.in, etc. You’ll be able to edit those to your liking, just make sure that when you upgrade you don’t miss any important changes in those files that may take place.
Dovecot will reference dovecot-sqlite.conf for user, password, and setup information. That conf file will reference a database in /etc/mail/vmail.sqlite. It’ll also have the dovecot-auth daemon listen for sasl auth inside a private postfix directory.
Postfix will contain a master.cf file that is full of references to other files such as: vmailbox-sqlite.cf, vmaildomain-sqlite.cf, etc. All of those configuration files will also reference the /etc/mail/vmail.sqlite file, each containing a different static query to gather the information needed for that option. SASL auth will be configured to point at the dovecot sasl socket inside of the postfix private directory.
In essence, all mail is controlled by the vmail.sqlite database, everything else is a static flatfile configuration. Since changes to the sqlite database don’t need to be postmapped, postfix won’t ever need to reload automatically. Same goes for dovecot.
To start with, the code will get a transaction lock on the vmail.sqlite database, drop everything, and recreate everything, and commit it all. Further optimizations will be made down the road to better support installations with thousands of email users.
The vmail.sqlite database will look much like what the ravencore database looks like now, as far as mail goes… a domain table and a user table. There may be more tables depending on how far I want to take the features at this point.
By cormander
I love yum. It’s yummy. Keeps my system updated and makes installing packages easy as pie, and it’s amazingly robust.
Like many things though, it does have it’s faults. This particular one annoys me to the bone:
primary.xml.gz 100% |=========================| 784 kB 00:01
http://fr2.rpmfind.net/linux/dag/redhat/el5/en/i386/dag/repodata/primary.xml.gz: [Errno -1] Metadata file does not match checksum
Trying other mirror.
primary.xml.gz 100% |=========================| 784 kB 00:01
http://apt.sw.be/redhat/el5/en/i386/dag/repodata/primary.xml.gz: [Errno -1] Metadata file does not match checksum
Exiting on user cancel
Well, this technically isn’t yum’s fault, it’s doing exactly what it should, not proceeding with a file that has a bad checksum. This particular case today is caused by the maintainer of one of the repos that I’m using, rpmforge. Since I was installing something that existed in rpmforge anyway, a quick disable on the command line does the trick:
# yum –disablerepo=rpmforge -y install package
This saves me the hastle of having to edit the repo file to disable it, and then re-enable it later.
Don’t get me wrong, I love rpmforge too. This just happens from time to time, is usually caused by them, and annoys the hell out of me.
Other faults of yum? Sure, but most of them exist in the realm of the people who publish their yum repos. Forgetting to update their repo after updating package, or not signing their RPMs correctly. I’ll admit that I’ve done this myself in my own repos I maintain for my own servers.
But here is a thing with yum that I dispize.. and I don’t know if it’s yum, or the python linked RPM library, but sometimes it hangs, and I have to send it a very nasty SIGKILL. Doesn’t happen that often, but damn I hate having to do that. Makes me want to crawl out of my skin.
By cormander
I found a patch for postfix to support sqlite. I downloaded it and built it with postfix 2.5.1 and did some testing and it appears to work just fine. Here is a link to the patch:
http://www.treibsand.com/static/files/postfix-2.5.0_sqlite.patch
I created a postfix package directory and posted the source RPM I built for it in there:
http://www.ravencore.com/packages/postfix/
It’s based off of the postfix RPM from fc9. Note that it requires a pretty recent version of sqlite3 to be installed on the system (more specifically, it needs functions such as sqlite3_prepare_v2 to exist)
CentOS 5 has sqlite-3.3.6 which isn’t recent enough; I rebuilt fc9’s sqlite package (version 3.5.6) and installed it on my CentOS5 system, it doesn’t appear to break anything that uses sqlite (like yum), so I went ahead and just added that version (or greater) as the build and install dependency. Down the road I might venture to figure out exactly what is the lowest version of sqlite you can get away with.
As noted in my roadmap for 0.7.x, this postfix package will be required down the road, and I’ll be maintaining it in a yum archive along with several other things for RavenCore.
By cormander
You may notice something a little different about the blog. If you don’t, they you’re either a new reader, or blind.
I spent some time and got the whole RavenCore website put into wordpress, and moved the directory down to the root of the server. There still may be some quarks to work out, please contact me if you see anything out of wack.
By cormander
Well after spending much of my day on merging version 0.3.3 with much of the latest development code in trunk, I’ve made some very good progress on putting out the first version in the newest “testing” tree of RavenCore.
Here is the feature list I specified earlier:
* use perl Net::Server for the daemon
* enhanced Base64 communication and error handling
* “express install” feature
* port mysql db backend to sqlite, change .shadow file to MD5 hash
* rework how mysql users and databases are managed to survive the back-end’s port to sqlite
3 of those are done. I now have to do the “express install” feature, and rework mysql users.
There are also probably plenty of bugs… hell I’ve fixed a ton already. This kind of thing happens when you try to merge a complete rewrite with legacy code. I even took the liberty of naming one of the files “legacy.pm” because it’s full of version 0.3.3 functions, and I’ve had to modify the hell out of them.
I could package up this code right now and release it under testing, it does actually work for the most part.. but I did say the 27th so I’ll give myself another day to regression test this bad boy.
By cormander
Wordpress 2.5.1 is out, and they recommend an upgrade. I was annoyed to find that they didn’t release a patch to upgrade from 2.5 -> 2.5.1… so I went ahead and made my own:
wget http://wordpress.org/wordpress-2.5.tar.gz
mv wordpress wordpress-2.5
wget http://wordpress.org/latest.tar.gz
mv wordpress wordpress-2.5.1
diff -ruN wordpress-2.5 wordpress-2.5.1 > upgrade.patch
Then I cd’d into my wordpress install and did:
patch –dry-run -p1 -i upgrade.patch
There were a few offsets, but no hunk failures… my hacks to wordpress have lived through the upgrade. Since this showed no failures, I went ahead and ran it without the –dry-run, and then logged into the admin section and clicked the “upgrade database” button.
All done. That wasn’t too bad.
By cormander
Here are some quick notes I threw together for a road-map over the next few weeks:
0.3.3 released this weekend (by 4/26/2008)
0.7.0 released this weekend (by 4/27/2008)
0.3.3 features:
* apply test patch
* set $rebuild_conf in rehash_named to 1
0.7.0 features:
* use perl Net::Server for the daemon
* enhanced Base64 communication and error handling
* “express install” feature
* port mysql db backend to sqlite, change .shadow file to MD5 hash
* rework how mysql users and databases are managed to survive the back-end’s port to sqlite
As noted above, these should all be done by this weekend. Looking forward, here are my current plans on features moving forward though 0.7.x:
0.7.1
* base engine to “install” things on physical virtual hosts which read a config
make a config to install wordpress
maybe make one for phpbb2
* narrow down the ravencore Makefile and have it call external install scripts
add “clean” option to Makefile
alter install scripts to use .patch files on 3rd party programs instead of crazy sed/awk commands
* httpd.include.tpl template files
one for the whole server
one per-domain
something like this: http://www.ravencore.com/forum/viewtopic.php?t=1016
0.7.2
* upgrade squirrelmail to 1.4.13
* upgrade phpmyadmin to 2.11.5.2
* upgrade awstats to 6.7
* upgrade phpwebftp to 4.0 (5.0 if it’s out by then)
* add email auto-responders, port from DBD::CSV to DBD::SQLite
More info here: http://www.ravencore.com/forum/viewtopic.php?t=1009
* global server aliases, default forward to server admin: abuse, postmaster, root
More info: http://www.ravencore.com/forum/viewtopic.php?t=1028
0.7.3
* alter how amavisd/postfix/clamd/etc configurations are built, use .tpl files instead of the one-file-per-line format that’s used now
* maintain custom version of postfix and dovecot; have postfix use dovecot sasl auth, have dovecot use sqlite auth; configure postfix with sqlite3 patch http://www.treibsand.com/static/files/postfix-2.5.0_sqlite.patch and store vhosts in there
* consider using amavisd-new’s milter with postfix
* consider adding functionality patched here: http://www.ravencore.com/forum/viewtopic.php?p=3661
0.7.4
* use geopt for rehash functions: http://www.ravencore.com/forum/viewtopic.php?t=1029
recreate the “bin” directory and use symlinks to run_cmd to use as aliases
* simple ACL for admin user to lax or restrict what the ravencore admin user can do
* implement the atomic-yum web GUI
0.7.5
* IP-based virtual hosting, SSL-certificate support per-domain
* add support for postgresql databases, add phpPgAdmin
0.7.6
* xen Dom0 host control support via libvirt
By cormander
I’ve sufficiently tested the grsecurity iptables-1.4.0 rpm built for centos 4, centos 5, and fedora core 8. Each rpm set is in each respective yum repository. They are built with the grsecurity kernel-grsec-devel and kernel-grsec-headers installed on the system.
I was also able to build the iptables RPM on opensuse 10.3, however that system doesn’t like the init script or the dependancies… it does chkconfig and init scripts a little differently. Damn system. Sorry suse users, no grsecurity iptables RPMs for now. Maybe a little later I’ll grab the opensuse iptables src rpm and use the 1.4.0 source and grsecurity-iptables patch, but for now, SOL.
Have fun with them. As always, they’re hosted in the ravencore grsecurity yum repo:
http://www.ravencore.com/grsec/
By cormander
With a grsecurity patched kernel, there’s a question that I haven’t given too much thought since I started maintaining grsecurity kernel RPMs: “Should I still have SELinux options enabled in the kernel?”
My answer so far has always been “remove it; I don’t use it anyway”. Well, I decided to give this some more thought. Well, that’s not fully accurate; the real reason I decided to give this more thought is SELinux options reappeared in my kernel after I merged fc8 options with it and I’m too lazy to turn it off and rebuild.
Just because *I* don’t use SELinux with grsecurity (thank goodness for the ability to disable it at runtime) doesn’t mean no one will. I’ve always thought of SELinux as the lesser when compared to grsecurity, but it does have a feature grsecurity does not; filesystem labels.
Not much use for me because well, I don’t use filesystem labels for security contexts, but it does have a time and a place. So SELinux, you’re back on my good graces enough to not get removed at compile time. I’ll keep you in there because I can turn you off anyway, and you still may be useful to someone else.
By cormander
I’ll admit, I’ve delayed the next release of RavenCore for quite some time. There are many factors for this, but I think the biggest one of all is the number of changes I’ve made in the latest development code. I tried to do too many things at once, and it snowballed into an avalanche of new features. It’s pretty much an almost complete rewrite of the entire code base.
Now I’ve decided to take a different approach to get the next release out. I’m prioritizing the new features as what should go first, and I’ll be releasing them one at a time. Here is what the next few weeks is going to look like:
This week I’m putting out ravencore 0.3.3, the maintenance release based off of the test patch I released a little while ago. I’ll then immediately copy the 0.3.3 version to the 0.7.x tree in subversion. I’ll then go through this list, and one by one, back-port these features from trunk into 0.7.x, and release after each one (or couple) is done.
A quick list of what to expect in 0.7.0 right off the bat:
* A much more stable back-end daemon. Not that 0.3.2’s is by any means unstable; the new code will be smaller, easier to read, use a standard perl lib for accepting connections, and will solve one of the long standing bugs present in the session handling of phpmyadmin.
* A 5 step install. “Install RavenCore in just 5 easy steps”…. going something like this:
1) download ravencore.repo file
2) yum install ravencore (dependancies all resolved automatically)
3) set admin password on command line
4) start ravencore
5) login to interface (with the password you set in step 3) and click “express install”
The “express install” will basically ask no questions, it’ll assume all defaults. The other option to “express isntall” is “advanced install”, which is more or less the current install method.. you get to change the variables, among a few new things.
* The back-end database will no longer use MySQL, it’ll use SQLite. The structure won’t change at all (that’s to come in version 1.0) but there are a number of benefits to this which you can read about in the “Development Highlights” section of the wiki.
Once this is all final I’ll make a more official post in the news section of the site containing this (and more) information.
Some features from trunk can’t be ported because they change too many things – so there will be a point where 0.7.x will have changed enough to suite most people, it’ll become the “stable” version, and development of version 1.0 will be kick-started once again with a lot less work needed to be done to finish it.
