Midgard2 - flexibility rocks

Posted on 2007-01-11 23:47:33 UTC.

There's much work done ( and much to be done still! ) to get clean and clear Midgard2. However I am quite impressed when I think how things look like now.

Core is just yet "another" library and proved to be quite usefull for:
language bindings, GTK applications , Apache modules or command line applications.
It's just the way how library should be built. Ah! I am very glad I can use midgard-core with libgda ( website seems to be refactored now ). At last I can forget about MySQL dependencies and its API.
As an addition, I also added openssl support today.

Apache module is just yet "another" Apache module. Unlike Midgard1 module, this one is "environment and language unaware". And it means that this module can be used with any language like perl, java or python.

And...this module forced me to create that "trick" in core , so you can initialize midgard library in GType system unaware application and use its power with forked GType aware child.

Today, new refactored midgard-apache module finds midgard hosts ( those prefixed too ) very well. Correctly parses uri to get midgard pages and nicely serves images attached to pages. This is very nice to notice correct 200, 404 or 403 :)

PHP extension is just yet "another" PHP extension. You can use it with php-cli scripts or just build websites.
( with MidCOM you do it quickly :)

Repligard functionality ( known from Midgard1 ) is just "yet another feature" of midgard-core.

I must say. It looks good :)

And what happens with stable 1.8 branch? I fixed few important issues related to midgard_collector, so we should have new 1.8.2 released very soon.

Midgard sitegroups

Posted on 2007-01-17 13:18:06 UTC.

Midgard sitegroups concept is well known since Midgard 1.4 release.
This excellent feature has beed added to Midgard library almost seven years ago but till today no one clarified few important issues. I have been talking with Bergie and Rambo about this and we decided to clarify and fix this once and for all.
However you may expect some issues raised in new Midgard releases, you should not to be worry about them as it will affect only Sitegroup 0 Administrator , well known as Root User.

BTW, I updated midgard sitegroup concept docs and we should definitely use this convention to name Midgard persons:

• Root user ( Sitegroup 0 user )
• Administrator ( a person who is an administrator within sitegroups X )
• User ( any sitegroup X user )

Sitegroup issues which now are clarified:

• You can change object's sitegroup only when you are in anonymous mode or you are being logged in as Root user. And you can change sitegroup only ( and only ) when objecty is not yet created. In Midgard 1.8 series any attempt to change sitegroup property is silently ignored ( due to PHP&ZEND1 ).In newest Midgard , any attempt to change object's sitegroup will trigger unconditional warning.
  

• Administrator and User always manage objects in sitegroup context.

• Root user always delete, update or create object in sitegroup context. In this case it means that we use object's sitegroup property to determine its sitegroup instead sitegroup of current application. Unlike Midgard1 concept ,it means that being logged as Root user you always update or delete object with particular guid and particular sitegroup.
  

• Root user can query objects ( using Query Builder ) and get many objects identified by the same guid but within different sitegroups.

The issues I know for now are:

• how to handle midgard_object_class::get_object_by_guid($guid) static method in a safe way? Probably we should add second optional argument ( sitegroup identifier ) valid only for Root user.
  

• how to handle midgard_replicator::export_by_guid($guid) static method? The solution seems to be tha same as this proposed for get_object_by_guid.
  

For now , midgard-core tries to be as smart as possible and if one of these methods returns more than one record ( for Root user ) MGD_ERR_SITEGROUP_VIOLATION ( Resource link crosses sitegroup borders ) error is unconditionally thrown.

And as we work on new replication framework , we should be able ( very soon ) to replicate the same object between many sitegroups in one database.

Setting up Maemo scratchbox

Posted on 2007-01-20 15:20:08 UTC.

It's worth mentioning. When you try to setup scratchbox for maemo you probably ended up with googling and looking for this :

Xephyr cannot open host display. Is DISPLAY set?

I am almost sure that you use gnome, so enable TCP connections in GDM setup.

Unfortunatelly I resolved this weird issue while my user's scratchbox account started to acting up so obviously I need to reinstall scratchbox.

Midgard DB indexes

Posted on 2007-02-04 11:01:23 UTC.

I just realized that Midgard blobs table doesn't use indexes for legacy db structure. If you still use legacy applications and legacy code, add two indexes to blobs table:

CREATE INDEX blobs_pid_idx ON blobs (pid);
CREATE INDEX blobs_ptable_idx ON blobs (ptable);

Also increase cache size ( in my.cnf file ):

query-cache-size=92M

I set 92M in my case , but you can tweak it with different value.

in other news: It's not fun when the whole family is ill. Ladies are getting better. Ola must stay at grandma's place for one week more. Aga had 40,2 temperature two days ago and I got throat infected. It's just the way when your kid "brings" something from kindergarten ;)

Also I need to fix few minor issues found in 1.8.2. And make midgard-core 1.8.2.1 hotfix release package. I think I should do it soon as I am no longer a house nurse ;)

maemo.org, website which ....

Posted on 2007-02-10 14:10:29 UTC.

...sucks a lot.

Consider this real scenario:

1. You want to develope new application for OS2006
2. You need to update OS2006
3. You need to update maemo2.1 to maemo2.2

Maybe I am stupid , but clever guy do:

1. Open maemo.org
2. Follow "Maemo 2.2 'gregale' is released" link from frontpage
3. Follow one of these two links:

• The latest OS 2006 Software Edition is available here.
• For more info please see the releases page

1. Download latest OS2006 ( you completely hae no idea what to do with this OS )
2. Follow release page
3. Follow Maemo 2.2 'gregale' Release
4. Follow any link in 2.2 release section
5. Try to figure out what to do.
6. Count from 1 to 10 to calm down....
7. Imagine "I feel lucky"
8. Follow "wiki" link in main navigation
9. Dada! There's howto for developers!

There's 'Developers' link in main navigation. Why thi showto is not linked directly from developers section?

What does 'Wiki' means in main navigation? A Page that uses wiki? A tool? What? I wonder if I should create 'PHP' link on my website because website uses PHP...

Almost full quote from Vabank:

Q: Do you always keep money in a bathroom with your dirt panties? A: Yeah , sure! And I keep my toothpaste in my socks!

A good lesson for me being also a developer and release manager. Always use links and titles which describes what is the content they provide. Never use links with names which describe what is the tool they use.

I remember I lost quite some time few weeks ago when I wanted to find applications' catalog on maemo.org. Wiki page was the last one I selected.

A similiar scenario can be repeated on http://gnome.org website. This is impossible for end user to find anything. In the same time, my wife is trying to find some gnome application. Obviously, not being gnome developer, she has no idea that http://gnomefiles.org can be found at gnome.org > community > software map.

Start here

Posted on 2007-02-12 10:33:17 UTC.

Just two days ago I wrote about being confused when I tried to follow Maemo documentation. I looked at other websites addressed for end users and developers and noticed that Maemo is not the only one which provides unlogic navigation. Also Midgard ( the one I write docs for ) project website needs more love.

This would be cool IMO:

"Get started" section with:

• What would you like to do.

  • I have device XXX and I want to install new applications
  • Where...

  • How ...

  • I have device XXX and I want to update OS I have installed

  • Where ...

  • How ...

  • I am a developer and I want to write new application

  • How to start ?
  • What should I install on my device?
  • When and why do I need SDK on my device ?

• Features

  • What is ...
  • How ...
  • Why ...

The main point here ( and everywhere ) must be to treat everyone as a user. Even a developer. A developer is only a human and should have a good place where can start. It's not funny anyway when you prepare for hacking but first you must hack docs and notes.

A cool feature for any website with documentation would be possibility to bookmark resources the same way how you can add products to cart in any online shop. Once you like the website you can customize your start page and limit information only to this you are really interested in.

Probably I could contribute and write some simple howtos for maemo. But the real problem is the fact that the more I read docs the more I am confused. There are plenty of "how to ..." but at the same time there is almost nothing about why and when.

Example. There is quite long and "almost with details" how to create custom rootfs for device doc. But there's nothing about why do I need custom rootfs and why should I use it instead of downloadable and installable, ready to use SDK rootfs. What is more , I couldn't find any info why do I have to install rootfs SDK on my device :)

midgard-php or ZEND2 issues again

Posted on 2007-02-13 11:28:47 UTC.

I just created new thread on mailing.www.php-dev.

I started to refactor midgard-php extension , so we get better PHP language bindings. Current problem is "quite trivial":

<?php

$obj = new midgard_config();
$obj->read_file("my_config");

echo "Got configuration file for $obj->dbname database \n";

?>

This works very fine. But another line in the code:

print_r($obj);

just prints properties without variables being set. This is not important from application's point of view , but in "debugging" mode , it can be quite confusing.

BTW, if anyone has good url for zend-dev mailing list. Please let me know.

try midgard_article, catch midgard_exception

Posted on 2007-02-27 20:56:50 UTC.

What I focus on recently is midgard-core and midgard-php extension. I try to make both usable as soon as possible. Completely "forgotten" midgard-apache module must wait till Midgard core has more power. And midgard-data... I will write about it later , looks like many parts of midgard-data will be moved to core :)

Reading ZE2 sources and asking "too many" questions at pecl developers mailing list let me refactor midgard-php extension and implement it almost from scratch with new ( I must say , quite good! ) zend objects store functionality.

For the first time I felt that doing something with zend sources really does make sense ;)

What does it really mean?

• when you read php object property ( like echo $obj->prop ) you really read core's GObject property. So let's say you want to be smarter than core and want to "force" guid property value. No way, in such case core will produce warning which is propagated as php warning message ( but this is already known from 1.8 series )

• when you set value as php object property ( like $obj->prop = "abc" ) you really set core's GObject property

• when you destroy php object , you destroy core's GObject

• no more weird __res properties with even more weird integers which points to something or anything

For the first time I can say - this is language bindings.

And something completely new in midgard-php extension for pure OOP fanatics:

try{
    $ma = new my_article();
    $ma->get_by_id(1000000); 
} catch (midgard_exception $me) {
    echo "Couldn't get object by id. The reason is: ".$me->getMessage()." \n";
}

Message well known from mgd_errstr function is set as exception message, so what you expect here is:

Couldn't get object by id. The reason is: Object does not exist.

I just do not have article record identified by 1000000.

midgard_config or how to create personal database in 2 minutes

Posted on 2007-02-27 21:07:18 UTC.

Step 1.

Write simple php script, it may looks like this:

class my_config extends midgard_config {

}

$c = new my_config();
$c->dbtype = "SQLite";
$c->name = "midgard";
$c->save_file("MyData", TRUE);

What this script does is creating your own personal Midgard configuration. Second ( optional ) boolean parameter means "I want to write configuration in my home directory. Only there". 'MyData' is configuration name.

Step 2.

Write another script and it may looks like this:

mgd_config_init("MyData");

/* more own php code */

That's all. Your SQLite database is already created.

Stay tuned to learn more about "how can I create tables?" :)

midgard_config or how to create database tables

Posted on 2007-03-13 10:04:43 UTC.

Just two weeks ago I wrote about midgard_config and creating personal database. In that case I used SQLite provider and now I will try MySQL and PostgreSQL ones. If you used to switch between database types , I suggest to create few database types entries in configuration file.

Just like this:

Type=MySQL
#Type=SQLite
#Type=PostgreSQL

Comment , uncomment whenever you want. Yes, I know no one is so crazy to switch among database types every 5 minutes, but at least we have such possibility :)

I assume , you already created database and it's owner. So now it's high time to create basic tables and those ones needed for all Your MgdSchema classes.

Let's use personal "MyConfig" configuration file.

<?php

mgd_config_init("MyConfig");
$config = new midgard_config("MyConfig");

if($config->tablecreate) {

    midgard_config::create_midgard_tables();

    foreach($_MIDGARD['schema']['types'] as $classname => $data) {
        midgard_config::create_class_table($classname);
    } 
}
?>

Run this script and enjoy spring morning in meantime.

The things which happen in background are sexy or jazzy. In Poland you may say "kaczi" which means "ducky". ( Because these two twins has 'duck' in their surname ) BTW, if you do not know how to recognize them , the hint is: one is president and another is prime minister.

midgard_config::create_midgard_tables

Creates tables used by Midgard itself.

• repligard
• sitegroup
• midgard_user
• imports default SG0 root with 'admin' username

midgard_config::create_class_table($classname)

Creates all tables and columns needed for class.

• If class is multilingual then language content table is also created.
• guid and sitegroup column is automagically created
• metadata columns are also automagically created
• All required indexes are also created for table.

Now, you are able to login to Midgard database. Stay tuned to learn more about "how can I login in to Midgard database?"

In other news : hunt and recognize

drinking too much kaffe

Posted on 2007-03-20 07:51:30 UTC.

I tried to write simple midgard-java jni based bindings , but unfortunatelly got few not interesting issues.

If you follow almost every JNI specifications and docs and examples, you should run command like:

javah -jni org_midgardproject_connection

Not expected output is:

Failed to open object 'org_midgardproject_connection'

I was wondering what is the case as it worked perfect few days ago with HelloWorld example. And problem seems to be in some package being overwritten or removed or updated. I made few java related packages reinstall when I wanted to compile latest java-gnome. No success, though :/

Solution here is simple:

javah -classpath . -jni org_midgardproject_connection

Next problem is more difficult. When I compiled my org_midgardproject_connecion , I tried to run my new java script:

java -Djava.library.path=`pwd` org_midgardproject_connection arg

And exception:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Open at org_midgardproject_connection.main(org_midgardproject_connection.java:38)

Solution here is to "tweak" header and source file , which is badly created by javah tool.

JNIEXPORT jboolean JNICALL Java_org_midgardproject_connection_Open(JNIEnv*, jobject, jstring);

There are no '1' after underscores. So function prototype should look like this:

JNIEXPORT jboolean JNICALL Java_org_1midgardproject_1connection_Open(JNIEnv*, jobject, jstring);

After this change everything seems to be working very nice.

Aha, just in case if you would like to write midgard-java bindings. Compile it like this:

gcj  -shared `pkg-config --libs --cflags midgard` org_midgardproject_connection.c -o libmidgard-java.so

So now, java code looks like this:

class org_midgardproject_connection
{
    public native boolean Open(String name);

    private static native boolean SetSitegroup(String guid);

    private static native String GetSitegroup();

    public static void main(String[] args) {
            new org_midgardproject_connection().Open(args[0]);         }

    static {
            System.loadLibrary("midgard-java");
    }
}

And simple JNI C code:

JNIEXPORT jboolean JNICALL
Java_org_1midgardproject_1connection_Open
(
    JNIEnv *env,
    jobject _object,
    jstring _name
)
{
    g_type_init();
    MidgardConnection *mgd = midgard_connection_new();
    gchar *name = (gchar*) (*env)->GetStringUTFChars(env, _name, NULL);
    gboolean rv;
    rv = midgard_connection_open(mgd, name, NULL);

    (*env)->ReleaseStringUTFChars(env, _name, name);

    return (jboolean) rv;

}

I tried java script being logged in as custom user, and midgard-core tried to open configuration file from /etc. So I got "Permission denied".

Looks like it is working :)

300 Midgardians

Posted on 2007-03-31 20:03:33 UTC.

Just came back from cinema... you must see it!

Chicken or Egg

Posted on 2007-04-04 21:21:44 UTC.

As soon I as added midgard_config methods to create midgard tables I wanted to create new midgard_sitegroup.

( Midgard Sitegroup concept is well known but I still think I should write new mRFC. Especially now, when this class ( and simple&powerfull feature ) has new advantages. )

What should be done to create new sitegruop? Some simple method which just creates some record in database. OK, piece of cake, but to make this you need to know if logged in person is root. Hmmm... how can I check if logged in person is root if I have no such feature in core? That's clear, user accounts management should be written first. Simple...but wait. How am I to create user account in sitegroup scope if I have no sitegroup management?

It's not fun. Far from it. Writing two functionalities at the same time is not fun. To test them, you also need some simple environment. So writing two different functionalities and writing PHP bindings for them is far far from being fun.

Less fun is even if you think "it would be cool to reuse MidgardQueryBuilder for those classes". Not because one ( me for example ) may be lazy. But because the less duplicated ( or similiar code ) the less bugs should be resolved in a future. So, how can I use not MgdSchema classes with MidgardQueryBuilder if this one is highly optimized for them?

This question forced me to blessed refactoring. Review, tweak, tune, refactor, test, fix memory corruption and get the same with "new soul". And at the end MidgardDBObject class has been born from the will of having new midgard_sitegroup.

So now, class inheritance looks like this:

• GObject
  • MidgardDBObject
  • MidgardObject
  • Any MgdSchema class

or

• GObject
  • MidgardDBObject
  • MidgardSitegroup

This change brings more power, so now I am even able to reflect or introspect MidgardSitegroup class object and check if such class is multilang one. I do not have to hardcode it's table or property's column name everywhere. For such features I can use introspection. Now I can say: plain and simple :)

You should stay ( still ) tuned to learn more about "how to create new sitegroup and new user accounts". But probably I will add API documentation instead of blog entry. And as I said, new mRFC will be also nice. Not for voting, but for keeping specification and idea in one place.

In other news: Bergie in his blog mentioned 'offline vs online debate'. Yes, offline is wasting the time feature if you keep in mind services like GMail or any "typical" PHP based CMS. But what about CMS or framework which has independent logic library? One which is not aware if you work offline or online? Something like Midgard :) All you need to do is to focus on GUI. I think it's worth time and work and very good to have. All in all we can also start debate: do I need battery in my laptop or in any other mobile device if electricity is almost everywhere?

midgard_sitegroup

Posted on 2007-04-12 11:42:02 UTC.

I have already created midgard database and tables so now it's high time to create new sitegroup. New , installed database has already "built-in" Midgard Administrator account. It's done automagically so nothing to worry about at this point.

Log in to Midgard framework:

$user = midgard_user::auth('admin', 'password');

You can check if logged in user is root. But it's not mandatory as midgard-core will do it for you.

if($user->is_root()) { /* do something */ }

$sg = new midgard_sitegroup();
$sg->name = "My Test Sitegroup";
$sg->realm = "My Test Sitegroup Realm";

At this point we are ready to create new sitegroup , but this sitegroup will be private one. By "private" I mean it's not public so anonymous user can not see it or use. To make it public set its 'public' property to TRUE.

$sg->public = TRUE;

Create it.

$sg->create();

Sitegrup is created. We defined it as public so you can use it for example with login screen well known from Aegir. To list all sitegroups you can use list method.

$sitegroups = midgard_sitegroup::list();

This method returns array of sitegroup names. If you are anonymous user you get only names. Guid, realm and other data are completely hidden for you. But if you are logged in as root, you can get sitegroup object using it's name.

$new_sg = new midgard_sitegroup($sitegroups[0]);

Now, you can create new group and define it as sitegroup's administrator group.

$grp = new midgard_group();
$grp->name = $sg->name ."administrators group";

Keep in mind that it's last chance to set group sitegroup! As long as you are logged in as root, you can do this.

$grp->sitegroup = $sg->id;
$created = $grp->create();

if($created) {
    $sg->group = $grp->guid;
    $sg->update();
}

Stay tuned ( for a while only ) to learn more about "how to create midgard person account".

midgard_user

Posted on 2007-04-12 11:59:05 UTC.

midgard_user class is something completely new in Midgard2 architecture. This is something which completely separates midgard_person object from its account. User type , login , password its kept by midgard_user for which you can assign midgard_person.

Login and password is known for everyone, but there are two new cool user features.

• Person account can be active or inactive. You may have midgard_person record and its midgard_user record, but as long as person's account is inactive you can not login in.

• You decide what's the way to encrypt and decrypt your password.

Create account for person.

$person = new midgard_person();
$person->username = "John";
$person->lastname = "Smith";
$person->sitegroup = 1; /* replace it with something more suitable */

$person->create();

$user = new midgard_user($person);
$user->password("mylogin", "mypassword");

Third optional argument is ommited here. But if need you can use:

• MD5 hash ( default one )
• Plain text
• SHA1 ( not yet supported )
• PAM

PAM seems to be interesting here as you can define typical Virtual Host and use PAM authentication type for selected users only.

Once account is created, you may log in.

midgard_user::auth('mylogin', 'mypassword', 'My Test Sitegroup');

Sitegroup name must be set explicitly as another argument. Unlike Midgard1 architecture. To make things even more flexible, you are able to use fourth optional argument: trusted auth type ( FALSE by default ).

midgard_user::auth('mylogin', 'mypassword', 'My Test Sitegroup', TRUE);

In this step you do not have to define your hash type. If you want to change it, use password method.

New makedist with new Makefile

Posted on 2007-04-16 20:47:47 UTC.

I just commited new Makefile for Midgard so releases, nightly builds or any other quick snapshots should be build much much easier now.

The main idea is to use custom make command and define ( and export variables ) using command line or simple helper script. In our case makedist is a helper.

makedist

It has two approaches:

• define variables with default values

  MIDGARD_RELEASE_VERSION=$1 MIDGARD_MAKEDIST_ROOT=$PWD MIDGARD_CORE_PREFIX="/usr/local" MIDGARD_ALL_IN_ONE_PKG="Midgard_AllinOne-${MIDGARD_RELEASE_VERSION}" MIDGARD_ALL_IN_ONE_DIR="${PWD}/${MIDGARD_ALL_IN_ONE_PKG}"

• define release version

It can be set as makedist argument, but if ommited, version is determined from midgard-core configure.in.

Examples:

./makedist 1.9.1

In such case we force version number.

./makedist

And here we set every package version to this one found in :

AC_INIT([midgard-core],[1.9.0alpha1])

make

Typical Makefile, but more flexible for daily builds.

Examples:

Build midgard-core:

make midgard-core

Build midgard-php and remove files

make midgard-php
make midgard-php-clean

Build midgard-apache and Midgard_AllinOne package, clean AllinOne dir

make all-in-one-dir
make midgard-apache
make all-in-one-dir-bzip
make all-in-one-dir-clean

Not a big deal , but there is no need to write additional build scripts for different purposes. Now it should be easy to write only very simple helper script and call make with needed arguments.

midgard_php_cache

Posted on 2007-04-20 07:11:51 UTC.

That's it. I finished to write 'proof of concept` midgard_php_cache. The main approach is to reuse data which are not changed among requests or are changed very very seldom.

The basic code looks like this:

$cache = new midgard_php_cache();

This is constructor so there's nothing to elaborate about.

if(!$cache->key_exists("test_key")) 
    $cache->set("test_key", "test_string");

Here I check if key exists in cache , and if it's not then I add key's value. You can do 10000000 requests at this point, set method will be invoked only once. And that's the point: I do not want to create new data over and over again.

$v = $cache->get("test_key");

And finally I get my value, using ( and knowing ) the key.

I tested only strings as this is real cache's bottleneck. It must be duplicated when value should be returned. But if you consider the fact that during every request you must create plenty of variables which will or will no be used , it then seems to be a small detail.

If I could describe cache's features I would say:

• global data storage ( all data ara available per module )
• no need to select data from database or from files if you need to use the same data with every request
• integers or indexed arrays can be stored among requests without any need to initialize new data ( and thus without any need to allocate new memory for value containers )
• file contents can be reused from cache ( even with string duplication "overhead" ) reducing I/O activity to none.
• even as public data storage can be used for private ( or semi private ) data using unique key identifiers
• it's written using zend so stored data are pure, typical zend ( and thus PHP's ) value containers

midgard_php_cache VS memcache

Posted on 2007-04-20 09:57:39 UTC.

Time for a "benchmark". Let me know if you think the simple code I used is wrong.

I created two loops. 10000 steps per each.

for ($i = 0; $i < 10000; $i++) {

}

In first loop I used:

$_kv = "a".$i;
$memcache->set($_kv, $_kv);

In the second:

$_kv = "a".$i;
$memcache->get($_kv);

Next I replaced memcache object with midgard_php_cache one, and invoked almost the same methods. Simply I created 10000 arrays like this one:

"aX" => "aX"

Average time for both loops:

• memcache 0,9749
• midgard_php_cache 0,0525

Average time for only second loop:

• memcache 0,505
• midgard_php_cache 0,0369

In latter case I also printed values returned by both objects to make sure I get some data. Apache has been not restarted and I got expected values ( a1, a2, a3....) in both cases. Which looks like values has been cached among all requests.

I just wonder... why difference is so noticable?

Trying GDA on N770

Posted on 2007-04-30 08:42:18 UTC.

Last week I found Johannes Schmid blog entry about new libgda3 packages. I downloaded package's sources and recompiled it for gregale on N770. Well , I tried to recompile. These libgda3 package have some major issues:

• Can not use debian files with gda 3.0
• Sections in control files are incorrect

The latter case seems to be the most common issue I can found in debian package built for armel architecture. But maybe this is my fault. Anyway , I can not install package from file using AM. It always complain "Incompatible package" when package's section is for example lib, while it should be user/lib.

I have no idea how packages can be installed via AM directly from repositories as AM always is getting and refreshing packages list and then throws a warning "Can not refresh packages list".

Anyway , I need libreadline4 to install gda packages ( those ones I had to recompile myself ). Unfortunatelly, it seems that only bora supports libreadline4 and I need to build another packages for myself. I hope this one will be the last one. In any other case I am afraid I will have to compile plenty of packages for myself.

My big wishlist ( one wish to be exact ) is to have usable AM on N770. I have at least 4 or 5 repositories added. AM doesn't complain that repos' locaions are invalid, and I have no idea if those are good or do not trigger conflicts. Well... I should be happy anyway, as I have about 5 applications listed in AM that can be installed. None of these is the one I want.

Bread, electricity and Midgard2

Posted on 2007-05-01 09:30:11 UTC.

/usr/lib/pkgconfig/*.pc
/usr/lib/*.a
/usr/lib/*.la

gda-sqlite

gda-sqlite.install file should contain:

/usr/share/libgda-3.0/sqlite_specs_*.xml

Other issues

I have no idea why all sqlite spec xml files seems to be missed. Normally ( when compiled for i386 architecture ) gda library looks for spec files in lib directory and then in share one. When packages are compiled for armel architecture , gda looks for spec files only in lib directory.
To make libs working correctly I did this trick:

rules file

cp $(CURDIR)/debian/tmp/usr/share/libgda-3.0/sqlite_specs_*.xml $(CURDIR)/debian/tmp/usr/lib/libgda-3.0/providers/

gda-sqlite.install file

usr/lib/libgda-3.0/providers/*.xml

There are other xml and dtd files which should be correctly copied for debian packages, but there are only warnings thrown without them.

More beauty for Midgard or midgard-python in action

Posted on 2007-05-04 10:37:18 UTC.

Some day I had read about beauty and a Python. And yesterday I though "Why there's no Python's beauty for Midgard?" Python C API is very well documented so I couldn't wait any longer. I started with MidgardDBObject, MidgardObject, MidgardConfig and MidgardConnection base classes... and commited initial bindings.

About two months ago I wrote about creating Midgard unified configuration file and SQlite based personal database using PHP bindings.

Today I can do the same with Python.

import gobject
import midgard

config = midgard.config()
config.set_property('dbtype', 'SQLite')
config.set_property('database', 'midgard-test')

if config.save_file("midgard-test", 1):
    print "Configuration file created successfuly!"

Looks promising :)

midgard-apache, first requests

Posted on 2007-05-09 09:38:44 UTC.

First requests using new midgard-apache module :)

What's changed?

• Apache module initialize core's MidgardRequest object so it's available in midgard-php extension without need to write hacks.

It means that one should be able to write the same PHP application for command line and for web environment.

• $_MIDGARD global array has new key pages initialized as array with midgard_page objects.

All pages found from url are stored in this array, so there's no need to get page object during the request as it's already available. This array is always initialized for website. At least it always has host's root page available.

$_MIDGARD['pages'][0]

Always contain root page.

• midgard-root.php file is parsed after connection for request is established. Not before.

$_MIDGARD superglobal is available in any self defined root file. In the first line of it. You do not have to initialize your application from code-global page element. You can initialize it in file, before midgard style engine starts.

My simple test root file looks like this:

<img src="/spider-admin/spider_logo.png">
<?php
    foreach($_MIDGARD['pages'] as $page) {
        echo $page->name;
        echo " &raquo; ";
}
?>

To get my screenshot I requested page with such url:

myhost:8101/spider-admin/topic/article/1

And of course $_MIDGARD['argc'] is 1 and $_MIDGARD['argv][0] is "1". spider-admin, topic and article are stored in $_MIDGARD['pages'] as 0, 1 and 2 index.

BTW, with very first request I did:

print_r($_MIDGARD);

And firefox wanted to install missed java runtime to run included plugin. What? I realized that print_r also printed 'article' midgard_page's content which contain java editor...

That's fun , I found java editor in application which I use for years at the moment when I completely rewrite midgard-apache and midgard-php ;)

Midgard 1.8.4 is coming

Posted on 2007-05-17 20:38:16 UTC.

This week I fixed another very important sitegroup related bug in midgard-core. As current sitegroup rule for objects is clear , the issue I found raised very ugly problem. Class constructor unfortunatelly set object's sitegroup even if object has been fetched from database. "Fortunatelly" it only affected object created by root user, which happens almost only when midgard sitewizard creates new host.

Interesting, I found this issue solved in CVS HEAD ( aka 1.9 ), and never thought it exists in stable branch.

So 1.8.4 ( instead of hotfixes ) must be made , just because of this one issue and few minor found in midgard_object_class PHP bindings. The latter is interesting ( as others issues found in ZE1 ). When you write code in C , you must know when you return NULL and when ''. But ZE1 ( or maybe even ZE2 ) never tells you when '' string is "marked" by engine as NULL...

Ah, I could forget. Datagard handles database updates much nicer now. Randomly "not updated" database problem is fixed.

Debian users should love midgard packages when installing 1.8.4.

At last we have meta midgard package , so

apt-get install midgard

installs everything you need to have midgard running in few minutes.

Midgard-data from debian packages increases memory_limit in php.ini files ( 4 or 5 or both ).
After installing midgard you have brand new midcom-indexer package, which provides midcom indexer files and init.d script which starts automagically when package is installed. Needless to say. As soon as you create new website with midgard sitewizard, indexer is waiting for you.

Important note! Debian package uses /var/lib/midgard/indexer directory for indexer. If you already use it ( self compiled ), fix your paths and copy indexed db files to this directory.

I have been testing indexer with few java sdk. non-free sun ones, kaffeine, gij-4 and free-java-sdk. The last one seems to be the best choice , so it's installed as indexer dependency and indexer uses its files directly instead of /usr/bin/java for example which can be linked to other java binary in your case. And something obvious , you do not have to remember about lucene library anymore. I think midcom-indexer does all out of the box.

In other news. I try to make Midgard2 as cool as possible. And it looks like soon we should be able to install Midgard2 library together with Midgard1 one.

Midgard developers meeting

Posted on 2007-06-07 19:40:05 UTC.

Another Midgard developers meeting. Cool!

This year I didn't fly to Helsinki, but drove with Solt instead. The trip was long but very interesting though. Great opportunity to see Lithuania, Latvia and Estonia. Especially the last one as we could visit an old city of Tallin. This is very interesting and exciting fact - possibility to travel to many interesting places when you are involved in open-source projects. Like Midgard.

As usual, agenda was changed ad-hoc, but we had interesting discussions about Midgard features. Also many things has been done. The most important ones include:

DBUS in midgard-core. Once again we all agreed, it's a "must be implemented" feature, which will make Midgard yet more powerfull framework.

Alexander explained new GPLv3 licence model. This is very important in open source projects to be licensed with correct and particular license. Especially if you consider the fact that companies like Microsoft used to grant patents on technologies which was initialy released under GPL license. Technologies like MgdSchema which highly extends repligard's one.

Alexander also started to write experimental midgard-php extension's feature to cache already included PHP files. With possibility to also cache (transparently) code included from database.

Bergie and Tarjei did excellent work with creating new centralized system to manage Midgard and MidCOM projects with all their subcomponents. Together with Johan they set up new mailing lists. Amazing! It's one of the best parts of meetings - many people can focus on the same work at the same time. It's not easy to arrange something like this "online". Especially if you consider different time zones, different model of work, different habbits and duties. I must say: Great work guys!.

On Saturday we started Midgard Installation storyboarding. Midgard CMS part is not easy to install as other ones. But, on the other hand it's not just yet another easy CMS. According to our plans and ideas, it will still need root access to install system libraries only but the rest ( like Virtual Hosts or databases ) will be managed via simple command line tools or web browser interfaces. Despite the fact we have many work to do , we should make it reasonably easy and fast as new architecture is language unaware, so setup can be done on many levels with exactly the same logic.

On Sunday we visited Suomenlinna island. Can you believe it? Without computers and internet :) Well... I just turned on my N770 to show how's new midgard-core works on this mobile device. ( Sounds like I can have my personal Midgard in my pocket :) Funny thing , we have been there only couple of hours, but sun managed to burn my face. Remember, this is a place you must visit when you're in Finland.

Great thanks to our sponsors: Maemo, Nemein and FTC. We didn't run out of beer during late night sauna breaks :) ("Night" rather. There is still sun shining about 10pm in Finland. ) And frankly speaking you can blame yourself if you haven't been there. Great people, great time and place. And excellent company for hacking, suana and beer!

midgard-project.org and up and down

Posted on 2007-06-14 11:48:49 UTC.

As you probably noticed Midgard website was acting up for last two days. And to be honest I started to blame myself as I made database update using datagard just before server crashed. I started to wonder: "why it happened on our server when it works fine everywhere else?". Bad luck?

This morning I got shell access to server and stopped to disable services one by one and investigate what is really going on there.

Two hours later I noticed that Apache is starting to use 100% ( well , top said 116% few times ) of CPU every five minutes. Cron job? That was it. I disabled MidCOM cron services till MidCOM developers fix the issue.

We just need to find the issue quickly.

Flood

Posted on 2007-06-15 09:12:19 UTC.

Flood and facts.

• 30 minutes.
• city without river
• water falls ( not rain ) and hail
• something I saw the first time in my life

See other pictures. Watch videos.

Bread, electricity and events for midgard-php

Posted on 2007-06-18 20:17:21 UTC.

Just commited latest fixes to generic connect method assigned to all MgdSchema classes. To those ones only , but it's very easy to add such method to all classes registered in midgard-php extension.

A bit of code instead of screenshot.

Create your own callbacks ( functions or methods ):

function testconnect($id = 0, $udata = 0)
{
        echo "Callback for event (parameters: ${id} AND ${udata}) \n";
}

class EventTest {

        function EventTest() {

        }

        function FetchEvent(&$param = 0) {

                echo "Got event with ${param} parameter. \n";
        }
}

My test function and test method are very simple. But the point here is to keep them simple. Once you have midgard-php working on your development machine, you can create very complicated callbacks :)

Let's create instances...

$et = new EventTest();
$t = new midgard_topic(1);
$art = new midgard_article(1);
$_b = 345;

...and connect callbacks to object which emmits signal.

$art->connect('action-updated', 'testconnect');
$art->connect('action-updated', 'testconnect', NULL, array(1));
$art->connect('action-updated', 'testconnect', NULL, array(3, &$_b));

Third parameter is explicitly set to NULL, as connect method expects object so ignores it if it's NULL.

$art->connect('action-update', 'FetchEvent', $et, array('cool'));
$art->connect('action-update', 'update', $t);

We set method name ( instead of function name ) when third parameter is an object. In both cases, fourth parameter must be an array and first one defines which event ( signal ) we want to connect to.

And the "application's" code:

$_b = 678;
$art->update();

And finally, the output:

midgard-core (pid:21523):(info): midgard_article::update(...)
Got event with cool parameter. 
midgard-core (pid:21523):(info): midgard_topic::update(...)
Callback for event (parameters: 0 AND 0) 
Callback for event (parameters: 1 AND 0) 
Callback for event (parameters: 3 AND 678)

As you noticed. Callback can be any function, and any class' method. MgdSchema or pure PHP one.

Have a nice events!

Midgard2 and Oracle

Posted on 2007-07-03 08:37:39 UTC.

Yesterday I tested Midgard2 libraries with Oracle database.

1. Download deb or rpm files ( install them of course ): http://www.oracle.com/technology/software/products/database/xe/htdocs/102xelinsoft.html

2. Make sure you have libgda configured and compiled with Oracle support. If you configure on debian and already have oracle debs installed, configure oracle support ( for libgda ) like this:

   --with-oracle=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server

3. Configure Midgard configuration file. Define Oracle provider.

   Type=Oracle Name=XE Username=system Password=secret

By default XE database is created so make sure you have defined this database and it also exists in /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/admin/tnsnames.ora file.

1. Export ORACLE_HOME.

   export ORACLE_HOME="/usr/lib/oracle/xe/app/oracle/product/10.2.0/server"

If ORACLE_HOME is not set , you should get error message like this:

midgard-core (pid:5222):(WARNING):Not connected to the Midgard database Database XE@. Could not initialize Oracle

1. Start midgard application.

In my case midgard-schema has been used with TableCreate option set to true. Unfortunatelly libgda doesn't support table create spec file for Oracle provider so all I got from application is this error:

midgard-core (pid:5226):(WARNING):Midgard connection error: ORA-00942: table or view does not exist

, No:(942), src:gda-oracle:gda-oracle-provider.c:666, SQL: Not available midgard-core (pid:5226):(WARNING):Can not prepare create statements for table 'repligard. Missing spec. file '(null)'

Let's hope Oracle spec files will be added soon.

Midgard and Midgard2 on the same host.

Posted on 2007-07-07 21:03:59 UTC.

So, I told few guys I have Midgard 1.8.3 and Midgard2 installed on my laptop ( aka development machine ). Tarjei also asked me about how to do it, and I think that blogging about this is much better idea than answering few mails ( or dozens in a future ).

Problem is not easy to solve. It's not easy as it's not easy to have PHP4 and PHP5 loaded at the same time for the same Apache server. But if you have Debian installed, solution might be easy. First of all, Debian ships both Apache and Apache2 packages. Both PHP4 and PHP5. And Midgard fortunatelly provides two different library files:

• libmidgard9.so for Midgard
• libmidgard-2.0.so for Midgard2

As long as you compile other libraries or applications against correct library , you should not be afraid about nasty segfaults triggered by invoking the same function name.

OK, what exactly I did? Midgard-php module from 1.8 branch must be PHP4 compatible, so using pure PHP5 features doesn't make sense in application which is designed also for PHP4. Midgard-php extension ( PHP language bindings for Midgard ) fully supports PHP5 ( ZE2 ) , and only PHP5.

Solution is simple. Install php4 and php5 packages. Install debian packages for Midgard-1.8.3 with php4-midgard extension and compile Midgard2 ( from trunk ) with PHP5. This at least resolves cli problem. Install apache and apache2 debian packages. libapache-mod-midgard package can be used for Midgard, while midgard-apache2 module can be self compiled. The latter solution is not tested by me, but I think I could create "backward friendly" debian packages for developers.

Midgard2 library doesn't provide legacy midgard-config application, so conflicts may be seen in schema files. However, those will be not changed too much in near future. midgard-data package with datagard is also "forgotten" for a while in new Midgard2 architecture. midgard-schema supports almost the same config file and implementation is encapsulated in library ( two libraries in our case ). And even, if you need two different config files for the same Database, one may be loaded from system configuration dir ( /etc/midgard in most cases ) and another one from user's conf dir ( $HOME/.midgard/conf.d ).

While I am writing this, I wonder if this would be good idea to rename midgard-php extension .so file. This also could help developers who do not use debian daily. midgard-php extension provided by Midgard2 can be loaded ( via dl function ) as many times as needed without causing any issues.

Apache module and its environment seems to be an issue still, but one is not used to switch development environment every 15 minutes.

This is not real howto. It's only idea, how this can be made when you have debian installed. There are still many problems which must be resolved, but at least start point seems to be easy.

We won again ( aka Ragnarok preparation :)

Posted on 2007-07-17 10:42:47 UTC.

Needless to say, I started my Ragnarok preparation, and together with Aga, visited Battle of Grunwald reconstruction place last sunday (the latter site unfortunatelly reached bandwidth limit). Also ( unfortunatelly ) our camera died last friday, so we had to depend on mobile phones' built-in ones. Believe me, it's Canon and hitting it down the ground didn't help, even I heard it usually help with canon though ;)

We spent plenty of 800 kilometers in car , and you could know what it means if you know what driving in Poland means. Luckilly, we were able to be witnesses of reconstruction and visit Malbork with wonderfull 600 years old castle. Bergie told me that few his friends are in Grunwald and are active battle members. We were wondering if we could meet them somehow, but as soon as I took Aga's phone to take this picture I realized that I lost her, and can not even make a phone call because I have both mine and her phone. Can you imagine? Find one person quickly among 100 000 others? When we found ourselfes we agreed that it's impossible to find few guys in such huge crowd.

After two rainy weeks, we got excellent weather with sun burning our skins. If you're going to visit Poland and have possibility to be in Malbork, do it. Remember that you need at least 3~4 hours to see castle "in brief". Probably next year we will take Ola with us, but this year we had to check "logistics" possibilities of those two places. You know, 5 years old girl is not going to be "nice" when you stay in car 1~2 hours because of endless queues, so next time we need to have tents with us and stay on "Grunwald Fields" couple of days to also see "bohurts".

Midgard2 - Ragnarok preparation

Yesterday and today I changed default directories and some files used by Midgard2 CMF.

midgard-core

• Fully separated library files
• Configuration files stored in midgard2 directory
• Required xml files stored in midgard2 directory
• midgard-schema command line tool renamed to midgard2-schema

midgard-apache

• module renamed to __mod_midgard2.so__

midgard-php5

• extension renamed to midgard2.so

With this approach ( at this moment ) , every developer can install Midgard 1.8.x and Midgard2 and configure only module and extension files' names in midgard's httpd.conf and php.ini files.

Ah! Last but not least! N800 has been more than helpfull during this weekend. I updated debian files for midgard-core, so now there's architecture "autodetection" to build armel and i386 packages even easier.

A piece of "clean" history

Posted on 2007-07-22 22:17:38 UTC.

I just noticed new pictures uploded by Bergie, and I reminded myself some interesting story while looking at this photo.

Henry III of France unfortunatelly has been King of Poland, and fortunatelly here in Poland , he discovered something which is known as toilet. When he came back to France he ordered to build ones in Louvre. Also, in Poland he saw a fork for the first time in his life.

I couldn't believe it, but at least Polish wikipedia says so.

include VS mgd_include_snippet

Posted on 2007-08-07 14:35:16 UTC.

To make this test I used Midgard2 from trunk ( do not think to run it on production machines for a while ;) and PHP5 5.2.

I created 100 snippets named 1,2,3,4... in /AegirCore/argv0/browse/ snippetdir tree. And 100 files in /usr/share/php/AegirCore/argv0/browse.

Test was simple:

for($i = 1; $i < 100; $i++) {

    $s = new midgard_snippet();
    $path = "/AegirCore/argv0/browse/".$i;
    $s->get_by_path($path);
}

And

for($i = 1; $i < 100; $i++) {

    $path = "AegirCore/argv0/browse/".$i;
    include $path;
}

I ran script 10 times, and got average times:

• mgd_include_snippet: 0,0054 sec
• include : 0,0080 sec

Interesting thing is fact that second case ( with files included from filesystem ) needed the same amount of time which snippets used, when I defined full path to included files.

Is code loaded from files faster than the one loaded from database? Myth. Only easier to maintain. Yet... :)

midgard_setup is getting better

Posted on 2007-08-10 14:08:03 UTC.

Have to mention about this before weekend :)

midgard_setup, a datagard replacement is now able to create fully functional ( I hope so :) Midgard database. Example code is simple:

$s = new midgard_setup_cli();
$s->save_config();
$sd = new midgard_setup_database($s->midgard_config);
$sd->create_database();
$sd->update_database();

A nice feature of __update_database__ method is fact that it also tries to create tables in case if one doesn't exist for specific class. It also can be used in Apache environment without security risk, as it can create tables only for classes registered via MgdSchema.

Also, I remined myself that Midgard unified configuration file can be managed with midgard-python bindings.

Next step is to install pear packages.

midgard_setup_pear

Posted on 2007-08-17 11:42:53 UTC.

It's really nice to have another part of new PHP based Midgard installer. midgard_setup_pear is quite small class ( despite the fact it's small, it's not easy to implement with completely undocumented PEAR classes ) which is responsible to install MidCOM packages from midcom pear channel.

Fetch code from svn.

Constructor

Very trivial:

$msp = new midgard_setup_pear();

Finally, constructor must accept midgard_config object, but at this moment it's not important.

Discover channels

$msp->set_channels();

This method update pear.php.net channel. Add pearified.com channel and discover, if it doesn't exists in your pear setup. Add pear.midcom-project.org and discover it. And, at the updates channels to fetch most recent packages info. The main goal is to update channels, so you can invoke it anywhere and as often as you need.

Install Role packages

$msp->install_midcom_roles();

Installs pearified/Role_Web, midcom/Role_Mgdschema, midcom/Role_Midgardsql' and 'midcom/Role_Midgardelement role packages. Separate method is needed because every role package has additional config keys which have to be set. And of course this method does it for you.

If one of the package is already installed, midgard_setup_pear will inform you about this fact, and will go further a nice ellegant way:

MIDGARD SETUP: Role_Midgardelement already installed

Install packages

$msp->install_base_packages();

Installs all base and mandatory packages required to start using midcom ( including midcom itself ). This method also updates packages, so again , you may want to use it anywhere and as often as you need.

MIDGARD SETUP: net_nehmer_blog already installed
MIDGARD SETUP: Installing template_kubrick
warning: midcom/template_Kubrick requires package "midcom/fi.protie.navigation"
downloading template_Kubrick-1.0.4.tgz ...
Starting to download template_Kubrick-1.0.4.tgz (10,911 bytes)

Preferred state

You might also want to change pear's config preffered state.

$msp->set_state('beta');

Of course you should set this just after midgard_setup_pear object has been created.

I want more

So, when your script needs to install some midcom package, you just need to use __ install_midcom_package__ method.

$msp->install_midcom_package('net_nemein_reservations');

Keep in mind that this method also updates package if it's already installed and midcom channel provides newer one. You do not have to define channel here. midgard_setup_pear will do this especially for you.

There are things on heaven and earth....

Go to PEAR docs. Spent some time on reading them. If you find some method doc like this

void &someMthod($params);

do not wonder what this mehod returns. It just retuns void class instance... Yeah... void class....

midgard_setup_vhost

Posted on 2007-08-31 11:08:02 UTC.

Yeah, I was to blog about this one week ago, but we are going to have one vacation week so plenty of things must be done before it. Also me and Aga had no much possibilities to spend time together for last two years as she just finished logistic school ( yep , she also works ) and this summer we decided to spend every possible hour together :)

I mean, I just wanted to say I had no time to blog about midgard_setup_vhost ;)

OK, so here it is: midgard_setup_vhost.

Constructor

$vhost = new midgard_setup_vhost($&config, "www.site.com");

$config is an instance of midgard_setup_config, a simple class which holds reference to midgard_config , defined ui and similiar ones.

Set directories

$vhost->set_host_directories();

Creates all virtual host's directories like cache file or DocumentRoot. It is safe to call it as many times as you need, if one of te directories exist, vhost object will ignore it.

Enable Apache module

$vhost->enable_apache_module();

Checks if midgard apache module is included in configuration, and include it if it's not. It also creates midgard's httpd conf file and tries to detect if Apache is already listening on the host's port. If not , midgard's httpd conf file is edited to include Listen directive.

Create configuration

$vhost->create_configuration();

Creates configuration file and save it to default location. User defined one may be used if location parameter is passed to method.

This method internally uses midgard_setup_vhost_config class which is responsible to create nice and readable virtual host's configuration.

midgard_quick_setup

Posted on 2007-08-31 11:19:10 UTC.

The simplest and easiest midgard installation tool. The main idea is to act with default, sensible values and ask only mandatory questions.

To test it you must prepare for tricky builds.

Build midgard-core and midgard-php5 from 1-9 branch. Get all midgard_setup files from trunk.

The quickest setup is available in dg_quick.php script. Almost final script is available in midgard_datagard.php

"Almost final" means that this script must be wrapped by some shell script like datagard to allow transparently exit php script and start it again with quite different settings. Simple we can not load Mgdchema classes when php is running so we must quit itself and start itself again. Of course without asking the user :)

To make first initial midgard setup just do this:

php midgard_datagard.php

Certificate in Logistics

Posted on 2007-08-31 11:32:59 UTC.

I am very happy :). Aga has European Certificate in Logistics since today.

Let us know if she should send her CV :)

Getting fat

Posted on 2007-09-19 08:18:15 UTC.

I must say it. All of you who still used to smoke cigarettes. Quit it now! I do not smoke for last three weeks and I hope it's the last time I quit with this awfull addiction. Though, first days with morning coffe without cigarette has been not nice....

And I must tell you one thing.

It's not true that you getting fat when you are quiting....

Last but not least , I decided to use patches.

datagard or coffee break

Posted on 2007-09-21 11:18:39 UTC.

So, you wanted to test latest Midgard?

• Make Your favourite coffee or tea
• Download it from nightly builds and compile.

• Run datagard

  /usr/sbin/datagard

• Give MySQL's admin password when asked for it

  Preparing to create database with default admin account 'root'

By default, you may just press enter. Yes, you drink your coffee so no need to make anything much more difficult.

• Be patient, datagard is performing many tasks now.
• Mmmm... What a delicious coffee... Do not forget. Do not smoke while drinking

• Give hostname if asked for it

  Hostname to use ? (yourhostname):

• Press enter if datagard did the trick and suggested correct name

• Stop and start apache
• Point Your favourite browser to http://yourhostname

Current issue is style not being associated with page and host, and the fact that I can not find any style or page elements imported to database. But hey! It's not even alpha quality :)

Working datagard

Posted on 2007-09-24 11:02:43 UTC.

I wrote about datagard on friday, and today I finally got it working almost as much as I expected. I know you probably know this Midgard template , but this one is unusual because I did hit Enter key only twice during installation.

The first one to invoke:

sudo /usr/sbin/datagard

The second to confirm empty password used for MySQL's admin acount.

Still , I needed to stop and start apache, but this will be fixed soon.

dataragd and replication

Posted on 2007-09-25 09:35:09 UTC.

Did I wrote something new about datagard yesterday? Whatever it is , it is old ;)

So... what happened since last morning?

I added support to reload apache's httpd server when new virtual host is successfully created. It's obviously two command less when datagard stops.

Today, early morning, Bergie suggested yet another cool feature. Why to ask user for MySQL's admin password, if server might be not secured by this? Quite easy as an idea , but I found it difficult to implement. Mainly ( no no, only because of it ) because php's exec function doesn't fill user's array with all data which are returned by executed command.

Docs clearly states:

"If the output argument is present, then the specified array will be filled with every line of output from the command"

But this:

$retval = 0;
$output = array();
exec($somecmd, &$output, &$retval);

Doesn't work as you would like to expect. $output is always an empty array. Anyway, I decided to use wild "convention", and I try to create database without password and when it fails I ask user for password. Not nice , but better than nothing for a start...

Also I added validation checks for midgard_setup_config which initially tries to "detect" if every single constant or class exists or is usefull in our case. Most of these are checked during configure script, but few times I "managed" to run datagard which changed permission with empty user and group name. Though , those passed correctly during package's configure.

And most sexy part of last changes. Midgard database provides predefined list of languages, so I had no other choice and implement replication support :) midgard_replicator requires much more love, but some missed features can be work arounded easily.

It's sample script to create one xml for more than 100 objects.

$midgard = new midgard_connection();
$midgard->open("my_configuration");

$qb = new midgard_query_builder("midgard_language");
$ret = $qb->execute();

$big_xml = "<?xml";

$xml_root = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>";
$xml_ns = "<midgard_object  xmlns=\"http://www.midgard-project.org/midgard_object/1.8\">";
$xml_ns_end = "</midgard_object>";

$xml_data = "";

foreach($ret as $lang) {

    $xml = midgard_replicator::serialize($lang);
    $xml = str_replace($xml_root, "", $xml);
    $xml = str_replace($xml_ns, "", $xml);
    $xml = str_replace($xml_ns_end, "", $xml);

    $xml_data .= $xml;
}

$big_xml = $xml_root.$xml_ns.$xml_data.$xml_ns_end;

file_put_contents("languages.xml", $big_xml);

You can import this file with import_from_xml method, once it's written. And so, datagard does it to give you more than 180 ready to use languages.

Building Midgard 1.9 for maemo

Posted on 2007-10-03 10:27:25 UTC.

I should say trying to build instead of building it for maemo. Though libmidgard10 and libapache2-mod-midgard are already installed on my N800. Again, list of dependencies is the key for success.

I changed debian files ( rules and control ), so rules file generates control file correctly when package is built. For example when I build package for maemo a section is replaced with user/libs' and architecture type witharmelinstead ofi386` which is generated when I build package for x86 architecture

Big issue with packages builds is naming convetion, so it's impossible to create correct debian files ( mostly Depends: section ) for every architecture. Your package might depends on apache2-mpm-prefork, while someone called it just httpd for maemo. This package also includes header files, so you do not have to install apache2-prefork-dev, and also apr binaries, but weird thing is that it doesn't include apr header files.

Workaround for this is ( easiest I found ):

cp -r /usr/include/apr-1.0 /scratchbox/users/pp/targets/SDK_ARMEL/usr/include/

As maemo packages are built as debian packages, one could also keep any naming convention so apxs2 could stay apxs2 and apr includes could be defined for apr-1.0, and not for apr-1.

MySQL used everywhere on debian distros with version 5.0 is build with version 4.1 for maemo.

Those are issues, one might and can to resolve quite quickly. But the real problem I found with php5-fastcgi package. To build php extension I need header files ( those I can just copy ) and phpize tool. Lack of latter is an issue :)

I need to investigate it more. And for those interested I used these packages:

• https://garage.maemo.org/projects/mysqldatabase/
• https://garage.maemo.org/projects/apache/

And debian repository:

deb http://maemo-hackers.org/apt bora main

BTW, if you need to install php5-fastcgi in your scratchbox, you might to want to change default httpd port. I changed mine from 80 to 8080. Without it I got `Permission denied' when being logged in as fakeroot and "wanted to bind" to port 80.

"Maemo Application Manager" I do not like you

Posted on 2007-10-03 11:00:28 UTC.

Why?

If package's control files has:

Section: libs

is easy installable via repository, while it's incompatible while installed via file file manager with double click. In such case, package has to have:

Section: user/libs

Does it mean that when I double "click" (doubleknock) on package file I invoke different Application Manager?

midgard-php on maemo

Posted on 2007-10-03 21:26:29 UTC.

It's just today when I thought that phpize is a big issue when I tried to install php5-midgard and midgard-data package on N800. I thought that simplest solution could be the best and just copied all required files ( like phpize, php-config, and few others ) directly to my armel target host.

And now it looks like it's not best solution because php-config should clearly points to extension_dir for example and this is quite weird ( in debian sense ) in php5-fastcgi package. At least this package could provide php-config to save plenty of other's time.

Anyway, at last I finally installed all midgard packages.

Even tried datagard, but this one fails because of lack of pear:

And I wrote simple php script which loads midgard extension via dl and prints midgard version:

Midgard Query Builder - fresh breath

Posted on 2007-10-12 11:00:18 UTC.

Last few days I spent on redesigning and starting to refactor Midgard Query Builder. Already made work is initial but very promising. Mostly, I wanted to fix all remaining issues from 1.9 branch, but code itself clearly told me that fix is impossible, and refactoring is quite excellent idea.

I had to port plenty of code from trunk ( aka Midgard2 ), but for example 1-9 branch got midgard_metadata support in midgard_reflection property "free of charge".

So first of all there's quite interesting new method for midgard_query_builder: add_constraint_with_property. Unlike add_constraint method it accepts only properties as third parameter ( instead of values ), so comparison may be done on database provider level without any need to know what value is hold by particular constraint.

add_constraint_with_property("metadata_publised", "<", "metadata_revised");

or more complex one:

add_constraint_with_property("topic.metadata.published", ">", "creator.metadata.created");

And second cool feature is that constraints added to Query Builder soon will be "moveable", which means that any constraint added to Query Builder can be removed before or after execution and builder itself will be able to perform different executions depending on collected constraints. New constraints also holds "join" like information so once constraint is added it forces internal join constraints and remove them once, it's removed itself.

Such code:

$qb = new midgard_query_builder("midgard_article");
$qb->add_constraint_with_property("topic.metadata.published", ">", "creator.metadata.created");
$qb->add_constraint_with_property("topic.metadata.revised", ">", "up.contentcreated");
$qb->add_constraint("name", "<>", "");
$qb->execute();

generates this SQL:

FROM article, topic, person, article_i WHERE article.name <> '' AND
topic.metadata_published > person.metadata_created AND article.topic = 
topic.id AND article.creator = person.id AND topic.metadata_revised > 
article_i.created AND article.topic = topic.id AND article.up = article.id
AND article.id=article_i.sid

Midgard and Ubuntu and segfaults

Posted on 2007-10-25 19:43:53 UTC.

Few days ago, Andreas reported segfaults on Ubuntu. Today I built ubuntu getsy packages for latest Midgard and was surprised when I saw "Segmentation Fault" in apache's error log file. Just after I made first request.

Hold on, this is Midgard 1.8 stable release. I know how it's designed and written and in most cases it's the last part of software I could blame for acting up on my ( on any ) server. A bit shameless, but true.

Gdb and newly written test script clearly pointed to dba_open which I started to blame a bit earlier, and just few minutes ago I found this bug report.

I must check if db4 upgrade can help.

Updated (2007-10-26): There are two ways to fix an issue. One is to update MidCOM from trunk, as this version doesn't use php dba functionality. Seems to be easiest but not recommended for production server , especially when you are not midcom developer.

Another solution is to rebuild php packages with flatfile dba support. To make this just download php5 source packages:

apt-get source php5
cd php5-5.2.3

Add --with-flatfile ( debian/rules ) to COMMON_CONFIG.

dpkg-buildpackage -b

Then, install php5-cli and libapache2-mod-php5. Update midcom code ( in file: /usr/share/php/midcom/lib/midcom/services/cache/backend/dba.php ) :

if (in_array('db4', $handlers))
{
    $this->_handler = 'flatfile';
}

I know it's not best solution but the quickest I found as temporary solution before ubuntu maintainers will update php and dba packages.

libmidgard and debug symbols

Posted on 2007-11-09 12:51:47 UTC.

I just improved debian files for midgard-core so upcoming Midgard 1.8.5 release will provide new package among debian ones. midgard-core debian files describe new libmidgard9-dbg package which is nothing but libmidgard9 without stripped binary code.

Why do we need yet another package for midgard? It's simple. When something really bad happens, for example segfault or sigabort, you do not have to recompile midgard library to debug it with gdb or valgrind. Instead, you will be able to install libmidgard9-dbg package with apt-get.

As 1.8.5 is not yet relesead, I built debug package for 1.8.4.

More events in Midgard 1.9

Posted on 2007-11-14 11:52:30 UTC.

Last week I fixed loaded event issue. Well... I am not sure if I can say I fixed it because of its complexity. Basically it's chicken or egg problem, but at least I added action_loaded signal in midgard-core and added emit method for all MgdSchema classes.

Simply I implemented method which is registered as GObjectClass' one. PHP itself doesn't provide any events functionality, and semi event pure php patches provides method like 'RaiseEvent' , while GTK-PHP bindings ( which unfortunatelly includes GObject bindings ) doesn't implement it at all. It's not private method, it's public one so objects itself can emit a signal if programmer really want it. It's mostly matter of convention and application stability.

Midgard-php developers can do the same. Write own events' callbacks or emit any signal if it's needed.

Example code would lokks like this:

$qb = new midgard_query_builder("my_class");
$ret = $qb->execute();
$obj = $ret[0];
$obj->connect("action_loaded", "my_callback_function", $params)
if($obj->creator = $myself)
    $obj->emit("action_loaded);

More beauty for Midgard

Posted on 2007-11-19 13:01:28 UTC.

It was spring when I wrote about python bindings for Midgard. While then, it was very initial work, today, I must say we have something really working, just after few days of work.

So as of today, you can do:

• midgard_config

  • read midgard unified configuration
  • list all possible midgard configurations
  • write configuration

• midgard_connection

  • open connection by its name
  • open connection using specified configuration
  • set application's sitegroup context
  • set application's language context

MgdSchema objects are also implemented so those are automagically registered as python classes when module is imported.

• Added methods
  • get_by_id
  • get_by_guid
  • create
  • update
  • delete
  • purge
  • is_in_parent_tree
  • is_in_tree
  • get_parent
  • parent
  • list
  • list_children
  • get_languages
  • get_by_path

You can find test code in midgard-python directory. Of course, python bindings require midgard2 library.

Beauty and the Beast

Posted on 2007-12-04 21:55:59 UTC.

I am not sure which one is the beast and which one the beauty. But I think both midgard-python and midgard-core may be both :)

Python seems to be the second Midgard's scripting language now. Two weeks ago I wrote about initial usable bindings work, and today there's much more:

• Midgard Query Builder
• Midgard Collector
• Midgard Object Class ( pseudo class for MgdSchema classes introspection and reflection )
• Midgard Reflection Property
• MgdSchema objects' parameters

Midgard Replicator is not yet fully implemented for midgard-python because it simply doesn't exist in Midgard2 ( aka trunk ). Reason why it hasn't been moved there is fact that replicator uses language specific SQL queries while Midgard2 itself uses something which you could name "built in" languages support. Whichis nothing more like languages persistant cache in details.

You can take a look at midgard-python test examples and compile module if needed. And I think I should start count days to first Midgard2 alpha release, and first python applications for gnome or maemo :)

midgard_user_auth

Posted on 2007-12-17 12:11:41 UTC.

I started to make new Midgard2 auth features backward compatible. So, to make things smooth and transparent I added new internal function which makes legacy authorization and it's triggered by simple check, check if new midgard_user table exists in database.

The good start:

• database is created with midgard branch-1-9
• python is used as scripting language
• midgard2 library is used with test

And simple example code:

user = midgard.user.auth("admin", "password", "python-sg", False);
if(midgard.user.is_admin(user)):
    print ("User is an admin")

Output with method names info:

** INFO: user.is_admin(...)
User is an admin

I didn't make any database changes, so let's hope both midgard versions could be used together soon.