Piotras' blog > Archive > 2006-01-01 - 2006-12-31

Piotras' blog: Archive

2006-01-01 - 2006-12-31

MidgardConfigFile

Posted on 2006-01-04 18:15:48 UTC.

Yeah, MidgardConfigFile is the new configuration directive for midgard-apache2 module. Takes only one argument. Filename.
Full path is defined by prefix used while midgard-core was compiled.

In my case I compiled midgard-core with prefix --prefix=/usr , so this file is located in /etc/midgard/conf.d directory.

This is smallest Midgard vhost configuration I have ever used:


ServerName ppmobile
DocumentRoot /var/lib/midgard/vhosts/ppmobile/8888
MidgardPageCacheDir /var/cache/midgard
MidgardConfigFile piotras
AddDefaultCharset utf-8
Quite huge? ;)

I am not going to explain midgard configuration file because it has every directive described and besides every directive is self explanatory.

Interesting thing is that I had to rewrite configuration file "handler" and make it fully OO.

Advantages:
  • midgard-java or midgard-php bindings will be even easier to make.
  • Internal functions like mgd_connect or midgard_get_database_connection may be methods of MidgardConnection object and take MidgardConfig object as parameter. Notice that data like username or password could be hidden even for midgard-core API!
  • MidgardConfig object can be used as global runtime loghandler, Gerror handler, etc, etc.
Disadvantages:
  • none
( Except the fact that some day we have to drop midgard *mgd structure and change API a bit ;)

MidgardConfigFile automagically sets midgard-root file and Midgard engine.
Unfortunatelly this can not be done in midgard-apache1 module ( or I couldn't find DEFAULT_EXP_LIBEXECDIR for Apache1 ).

It's in the wall.

Posted on 2006-01-23 16:00:26 UTC.

It was almost 11 p.m. at friday when me and aga winded some burning. I started to knock every neighbour's door and ask if everything is ok. Yes, everything was ok.
Everywhere, except our place.
It's not ok when your home is full of smoke, right?

Few minutes after my phone call, firefighters were at our place. Four men was running up and down trying to locate fire's source. There was no doubt, the fire was already inside the building. They thought , the fire is inside the wooden floor.

One man asked another one to bring some axes and thermal sensitive camera. When he told me  that they need to chop our floor I asked Aga
to put some warm clothes on and get only the most important things from our place. She decided to get only some papers and few DVDs with  all pictures of our daughter taken since she was born.

"Luckily" one of the firefighters found fire's source. It was inside the wall.
In the flat at lower level...
I have never thought that one may use an axe so fast. And I have never thought that one may watch tv and be completely unaware that fire is so close.
Completely invisible.

Now all is fine , except the fact that our place is untouched , but completely useless because of huge stench. There is -20 ( -30 this night ) outside , so we can not ventilate our home...


midgard_reflection_property

Posted on 2006-01-26 11:33:06 UTC.

And here it is. Midgard goes reflected.

You should get latest CVS HEAD to try new midgard_reflection_property class.

MidgardSchema ( "oh my god , they killed MgdSchema" )

Posted on 2006-02-07 11:45:03 UTC.


It's going to be brief.

MgdSchema is no longer supported. Initial Midgard MgdSchema C structure is succesfully replaced  with new object oriented MidgardSchema and MidgardSchemaClass.

A few benefits:

  • Language bindings for schema initialization will be easy to write
  • MgdObject and MgdObjectClass share the same data with MidgardSchema
  • MidgardSchema data are global for midgard-core while private and invisible for midgard "extensions" like midgard-php.
  • Memory management follows GObjects' rules
  • Initialized Midgard Reflection is much easier to write and extend.
  • Application's schema can be "extended" from xml standalone files ( private ones ) or directly from directories.





What really happens with Midgard

Posted on 2006-02-20 13:52:29 UTC.

I, Midgard  Release Manager and Midgard core developer feel responsible
to share with my knowledge.

  • Midgard produces SIGABRT because code is afraid when Chuck Norris is close.
  • Chuck Norris is midcom-admin. Always
  • Chuck Norris is invisible on #midgard channel. Always
  • Chuck Norris never lefts #midgard. Never
  • Midgard segfaults after Chuck Norris' roundhouse kick
  • It wasn't typo , it was Chuck Norris.
  • You are wrong if you think that Piotras can fix bug. Only Chuck Norris can do this.
  • Chuck Norris never cries. He leaks
  • Chuck Norris installed Midgard once. Perfect Software - he said.
  • There's only MidNorr and ChuGard.

Łódź

Posted on 2006-03-01 12:04:27 UTC.


I just found interesting news article about Łódź.
Well, it's not quite new, but it brings a bit of hope that Lodz will be part of UE some day.

Lodz is the city I live in and where all latest midgard releases are made.
Łódź in English means a boat , but this is very funny as there is no river here ;)
You can even find 'boat ' in Midgard releases' names.

There are also some nice galleries with pictures of Łódź.


Construct a better OO'ed day

Posted on 2006-03-01 12:13:11 UTC.


Day shouldn't be so frustrating when you can construct Your midgard-php objects better , easier and faster.

I changed midgard-php objects' constructor a bit. Just follow documentation.



Midgard 1.8alpha2 and PEAR

Posted on 2006-03-04 15:00:08 UTC.


I am trying to install somehow Midgard CVS HEAD with pearized MidCOM.

At the beginning I was surprised that no pear packages is installed , so I changed a bit my local datagard script , to force it to exit before asking about Virtual Hosts. As the initial setup is quite fast now I created another one database so I was able to see what happens with pear packages. 

Installing PEAR packages...

downloading PEAR-1.4.7.tgz ...

Starting to download PEAR-1.4.7.tgz (280,249 bytes)

............done: 280,249 bytes

requires package `Archive_Tar' >= 1.3.1

PEAR: Dependencies failed

OK, that's not nice , but pear could handle this somehow.
And next line: 

Command 'pear.midcom-project.org' is not valid, try 'pear help'

What? The command was: 

pear channel-discover pear.midcom-project.org

Let's try help: 

pear help channel-discover

Command 'channel-discover' is not valid, try 'pear help'

OK, let's try to look for it: 

grep -re channel /usr/share/php/*

No results.

I am using debian and that means that I may have php4 and php5 installed , and if you consider the fact that pear script looks for php command which is a link to php5 by default, you may notice some not nice surprise with great segfault caused by using php5 cli with midgard php extension builded for php4.

Currently I have no idea how to solve all these problems.

updated: I tested it also with PEAR 1.4.6 , but with no luck

Midgard Reflection Property once again

Posted on 2006-03-27 14:33:27 UTC.

Why there's MidgardReflectionProperty when we had MidgardObjectProperty?
Reflection used in class name better describes what this class is designed for.
It's somehow usual now ( I know, one would say 'I do not believe!' ) that we have reflection property class documented already in midgard-core and in midgard-php documentation.

As always midgard-php part of reflection property was the worst part.

I needed to change this : 

$mrp = new midgard_reflection_property();

echo $mrp->description("midgard_article", "sitegroup"));

to: 

$mrp = new midgard_reflection_property("midgard_article");
echo $mrp->description("sitegroup"));

A well known "big deal" is a possibility to fetch GObject instance while using php object. At first I decided to use Zend's resources ( sorry , I do not provide any url to zend sites ). But the problem with Zend resources is the fact that resources' id (very internal feature IMO) is available on zend and php level. As there is no possibility to set own private pointer as a member of zend object , one is forced to add resource's id as object's property.
Now think about 200 php object instances and 200 integers, it's easy to change object's resource's property value from 199 to 10 ( for example ) and get something very unexpected.
And now think about objects which use some ACL features in "critical mission" environment and possibility to change any resource id...

Let me quote:
"Watch, but do not touch. Touch, but do not taste".

I think this sentence from "The Devil's Advocate" describes perfectly the sense of resource handlers.

I spent some time on reading php-gtk sources. (It's a good way to "kill" your time anyway when you work with Zend. Read plenty of source code and try to understand it.) I thought that I could find some good way to make a good wrapper and use 1:1, php and gobject objects.
No way. This extension uses 0 property as resources handler for gtk object. What happens if I change value 1 ( let's say some window object ) to 2 ( let's say some other widget object)? I expect bad things.

At the end I decided to write very easy objects "collection". A collection which holds 1:1 references for zend ( php in this case ) and gobject objects. The idea is very simple. I use two data types: GHashTable and GSList.
The first one is a collection of zend objects' pointers assigned as hash keys , and gobject ones assigned as values. If HashTable is initialized with NULL compare functions then direct equal ( pointers ) one is used.
The second one is used as collection of pointers to any gobject which was initialized during particular request. So at the end of the request , it's enough to iterate over list and free all objects memory. Plain and simple.

The only problem I can see is zend object pointer which can be "overwritten" during some loop. What really happens with this pointer then? If You know, or you just found zend docs about this, feel free to tell me ;)

Before Midgard 1.8 is out.

Posted on 2006-04-01 14:01:29 UTC.

Yes, I know. It's April the 1st. But I am deadly serious.

What is the most important for 1.8 release:

  • stability
  • performance optimization
  • feature enchancements

It's quite hard to write about all changes in some particular order.

MgdSchema and its "self defense" possibility.

Improved Midgard Query Builder and possibility to add complex constraints and orders.

Improved datetime types support for object properties and midgard_metadata class.

Derived class support for Midgard Query Builder on midgard-php level.

Recently, I updated mgd_auth_midgard function docs. The "old" docs described boolean value returned , while in practice person's id is returned. Such "small" detail...

And last but not least.
Upcoming 1.8 release is fully backward compatible.

midgard-php Apache free

Posted on 2006-04-30 18:02:01 UTC.


It's going to be short.
But one would elaborate a bit about such issue, though :)

Midgard 1.8alpha2 is going to be released soon, and this is going to be the first Midgard release with midgard-php module absolutely Apache unaware.

Midgard-php is a pure php module.

On debian, you may use the same midgard module for Apache1 and Apache2.

No need to recompile.
No need to configure this extension with plenty of arguments.

The only one need is a need to use Midgard ;)

Midgard on debian = more freedom

Posted on 2006-05-12 09:14:31 UTC.


I started to debianize latest Midgard 1.8alpha2 release and during all the changes I made I realized that debian users should get more freedom and more flexibility while using Midgard Framework and Midgard CMS with this particular distro.

Debian policy doesn't allow to include symlinks like `libmidgard.so` is library package. Package name must be renamed following SONAME , so new midgard-core debian package will be named libmidgard9. We can include only those so files and ( symlinks ) which contain library version in name.

In theory ( you know how practice always looks like ;) I should be able to update my production server to Midgard 1.8 using Apache1 server , and use Midgard 1.9 development release branch with Apache2 and PHP5 on the same server without any need to break some setup.

In the other news. I can not stand it any longer. lintian complains about missing manual pages in debian packages for midgard , so I started to write those.
I should commit manuals to CVS repository soon.


Midgard2 started here

Posted on 2006-05-19 14:40:10 UTC.


Ragnaroek starts here. It was two years ago.

Midgard2 started today.

Midgard-schema, command line tool uses Midgard2 API , including MidgardConnection handler object and MidgardConfig object. In such shape it's going to be included in upcoming Midgard 1.8 release.

To make life more easier I wrote first (initial) official midgard-core docs.

Debian sid libc6 and Midgard with gcc4?

Posted on 2006-05-26 09:36:48 UTC.


Sunday. May 21st.
I run this sweet command on my Debian sid system. 

apt-get dist-upgrade

blah blah blah 

Replacing package libc6... Segmentation Fault...

First thing you can think about is mandatory ' WTF? ' .
dpkg segfaults, vi segfaults , cat segfaults , ls segfaults , everything segfaults ( well, at least other applications segfault ;) . Force reboot... System doesn't run at all...

Second thing you can think about is second mandatory 'F***, WTF?'.

In such case I could move hard disk to another machine and copy system partition , but with notebook...? Thanks god, I created additional empty partition when system, was installed for the first time. Installed sid again , mounted home partition to work with the same configuration and preferences and started to install midgard. Obviously , sid provides gcc4 and only MySQL 5 server and client, so Midgard ( obviously again ) can not be much usefull. Can not , but could.

I was too lazy to reconfigure system to compile everything with gcc3 , so I decided to fix this "should be resolved months ago" issue, and luckily found few lines of code which should be fixed.

The conclusion is simple. If the software you are working on has fundamental issues, then you should think about dist upgrade ( especially on debian sid ) , and with a bit of luck you can get unstable useless system. Later, you can fix basic issues.

I think , I should close this.


New cool Midgard features

Posted on 2006-06-14 17:08:09 UTC.


PAM authentication

Midgard is able to authenticate users with PAM module for years.
But sometimes you can not make it more flexible. Let's say you use two or three virtual hosts for the same sitegroup. And to make life more difficult every host should use PAM module , but with different configuration. One should authenticate via LDAP , another one via Domain Controller and another one via passwd file ( yeah , I know last example is mad if you consider how Apache handles files permissions ;).
With Midgard virtual hosts you could use MidgardAuthType directive and set its parameter as "PAM". With new "soon to be released" 1.7.6 release you will be able to define PAM service name.

Service name is nothing more like filename located in pam conf.d directory. Usually it is `/etc/pam.d`.

a. Create pam service file 'midgard_active_directory' 

auth  required /lib/security/pam_winbind.so

b. Configure vhost 

MidgardAuthType PAM

MidgardPamFile midgard_active_directory

c. Restart Apache

Midgard Query Builder

With Midgard CVS HEAD you can query objects and objects' parameters and attachments. Paramaters and atachments are exceptions for Query Builder. The same as metadata is.

To add constraint for parameter use magic "parameter" word and dot followed by property name. 

$qb = new MidgardQueryBuilder('midgard_topic');

$qb->add_constraint('parameter.domain', '=', "midcom");

$qb->add_constraint('attachment.name', '=', "xml.gif");

$qb->add_constraint('metadata.created', '>', "20010101");

$qb->add_constraint('up', '>', 0);

$qb->execute();

Configuration File

New unified configuration file now can be located in system /etc directory or just in user's home directory. Create config file: 

~/.midgard/conf.d/my_config


And load midgard application. Main configuration file will be ignored in such case.

Collecting good conventions

Posted on 2006-07-07 12:10:31 UTC.


MidgardCollector implementation is in progress. Briefly I can explain what collector is ( or what should it be ) for those who do not know it exactly.

MidgardObject methods gets all object's record and its data from database unconditionally. It means that all you can do is get particular record identified by some id or guid.

MidgardQueryBuilder is able to select objects' records from database and use or developer is able to limit selected records using constraints and thus can fully "interact" with 'WHERE' part of the SQL query.

MidgardCollector is something much more flexible. User or developer is able to limit 'WHERE' part of SQL query but additionally is able to limit 'SELECT' part of the SQL query. What is more important , selected data in collection can be overwritten via collector's API so the whole collection can be set "manually" by human or by SQL query.

I knew that MidgardCollector implementation is going to be hard , but I didn't thought that is going to be the most difficult part of midgard-core since new MidgardSchema technology has been announced.

1. MidgardCollector extends MidgardQueryBuilder. Well... easy to say if you consider that first QB implementation is not GObject's OO and the whole QB internals is shared in 5~6 files without public members, so trying to keep encapsulation of "class" internals and sharing public data means: refactor me.

2. Make yourself confused if you want to follow GObject's examples from manual and code conventions used for GValues. There is important difference between : 

GValue value = {0, }; /* convention */

and 

GValue *value = g_new0(GValue, 1);

The first one "initialization" end up with voodoo segfaults if you try to manipulate "the pointers" here and there and additionally provide them for PHP language bindings.

3. Make midgard_collector class derived one for PHP language. Especially when C structure of midgard-core is implemented as MidgardQueryBuilder PHP class breaking convention which says "do not use CamelCaps" for midgard classes. Well... I added midgard_query_builder class for midgard-php and thus I can easily follow midgard-core there and keep consistent logic for class names.

4. Zend conventions "rules". I wonder how many zend developers made me a target for a shotgun if I have never said anything good about zend.

a. Register internal class for PHP using Zend.( Please guys! Never mix up php and zend names! )
API provided by Zend developers requires hardcoded! class name passed as argument because the API in this point uses sizeof instead of strlen. So simply forget about dynamic class names pointers used through this part of API.

b. I am awfull right now , I know it:
There is no "Zend manual" part on php website. Instead you can follow excellent link "Hacking the core of PHP". "Thank you!"

c. PHP5 ( language based on Zend2 engine ) comes back to old convention to use CamelCaps for function and methods name. I know , one can use CamelCaps with PHP4 , but this is nothing but php developers ( or maybe only pear ones ) convention , because Zend doesn't use upper cases in names , forcing every single string to me lowercased. Try to be so smart then and use black magic and get class name or method name if what you can get is MidgardQueryBuilder or MiDgArDqueryBuilDER. One may use g_ascii_strdown all the time , but it's not the point to remember about this. Especially when midgard community already has been affected by such API change using PHP 4.x branch.

And hey! Last but not least! While beeing coach potatoe , try to be a monkey for a few minutes when you can follow monkey guys trip every evening :)

Aha! I could forgot. Seems that deathmonkey race time has some additionall 15 hours included.

midgard_language

Posted on 2006-07-24 16:54:06 UTC.


While discussion about midgard multilingual feature is in progress I added new midgard_language core's class.

Language class is defined as MgdSchema class so it's fully usable with Midgard Query Builder and sitegroup 0 admin can update languages or create new ones.


midgard_language has predefined 51 languages following ISO 639-1 language codes and names. If you want to update some language use midgard_language.sql file located in midgard/src/data/midgard CVS subdirectory.


Every language object has defined id property to use almost deprecated now mgd_set_lang function which requires id as argument. As soon as we release 1.8 stable , app's languages will be settable via $_MIDGARD object ( not array ) with its corresponding guid. I think guid is best choice as we can switch to ISO 639-2 or 3 in a future.

For those who would like to test midgard_language: 

$qb = new midgardquerybuilder("midgard_language");
$qb->add_constraint("code", "=", "en");

$ret = $qb->execute();

if($ret) 

    mgd_set_lang($ret[0]->id); 

$article = new midgard_article($guid); 
$article->content = "Some only english text";
$article->update();

In other news: midgard_collector is already implemented in midgard-core and PHP bindings for midgard-php are ready as well.

Multilingual midgard_topic

Posted on 2006-07-24 20:44:05 UTC.


Just wrote about midgard_language. So , while beeing in multilingual mood I updated midgard_topic so this class uses mulitlang Midgard feature now.

Added new multilingual title property.
Moved extra and description properties to lang table.

Well... I just reminded myself how many work had to be done with Midgard 1.4 to provide new feature , class definition and SQL queries needed for them.

With almost new Midgard2, such changes are... trivial.

Thoughts are in the air :)

Posted on 2006-07-29 14:04:50 UTC.

Tigert just mentioned about Midgard for Gnome desktop.
Bergie mentioned about GnomeGard project many many many months ago.
And I , for last two weeks think almost about nothing but some Gnome desktop application which could be used as simple Midgard Object's editor.

I am tired with editing css files via web browser. I am tired with editing code via phpmole.
As plenty of code , especially style's one resides in Midgard database , I can not use any simple text editor. WebDAV support is far from being perfect and usable.

I just installed Glade with the basic simple idea:

* Browse midgard classes
* Browse Midgard objects
* Use midgard_reflection_property to define object's property edit widget.
* Connect with Midgard API ( no need to configure Apache vhost for this )

To be shameless. Midgard core API in current state , today, is ready for Gnome desktop.
Even if such aplication is going to work in a year, I am going to write it.

MOB mock-up

Posted on 2006-07-30 14:05:12 UTC.


As soon as I installed glade I created new project. Compiled with midgard library and used few widgets. In this mock-up you are able to select configuration name from the list. Connect to Midgard database and see all MgdSchema classes in GtkTreeView widget. And of course display about window :)


It's enough to install midgard-core and dev packages ( libmidgard9 and libmidgard9-dev in my case ) and use it with glade. No hacks , no workarounds. Just easy as it should be. Just yet another gtk application.

MOB - objects' tree

Posted on 2006-08-02 11:42:42 UTC.

Just few days ago I wrote that I am going to write midgard-gtk object browser , even if I am to write it in a year. Today I have registered class list and objects' tree browser. Limited to root objects only, but I it will change within day or two.

A lesson from this part of MOB is that midgard misses some core functionality ( mostly object's method ) which is able to return only root objects if object's class is registered as "content tree" one.

So to build object's tree we should have such three methods ( two of them are already implemented ):

  • list_roots
  • list
  • list_childs


An interesting ( built in ) feature for tree widget is search functionality. If the tree has too many entries and you would like to find particular one by name , it's enough if you start typing. Exactly the same as you used to do using Firefox or Nautilus.


At this point I found one issue. Double clicking on midgard_snippet just hangs application. Or at least it looks like as I had no time to wait till it makes anything. Of course I selected all snippets. In such case it may be more than 5 thousands records.


Another interesting lesson from MOB writing is the fact that core misses config file functionality which should return only the names of configuration files or even better all the names of all configurations. Such approach would encapsulate real file names and could provide more readable configurations and their "destinations" , like :

  • midgard ( my.remote.host )
  • midgard ( localhost )
  • midgard ( my.another.host )

To be continued soon...

Aegir and PHP5

Posted on 2006-08-03 11:26:39 UTC.

If you try to run Aegir with PHP5 and you get this warning: 

Parse error: syntax error, unexpected T_ABSTRACT, expecting ')'  

in /var/cache/midgard/bergietest/1-5-5-0.php(991) : eval()'d code 

(5) : eval()'d code on line 538

Edit /NemeinRCS/xml_rcs_functions snippet:

Line 538 is: 

abstract =>  "CDATA",

Replace it with: 

"abstract" =>  "CDATA",



MOB needs name

Posted on 2006-08-07 12:42:44 UTC.


This weekend I spent in bed. "Nice" two days with MOB's code and heat. This is some kind of irony when temperature outside is going down after very long and and very hot July and my head's temperature is going up :)

MOB , Midgard Object Browser became a quite usefull application. I was afraid that snippet's listing hangs mob so I added progressbar and now I know it's just takes too much time. I didn't wanted to keep reading log files for successfull or failed operations , so decided to keep track of any Midgard error in window's statusbar. Now , mob's features are:

  • browse registered classes
  • browse object's tree
  • create new objects
  • edit existing objects
  • delete exisiting objects

I think it's high time to create Midgard subproject for mob, but mob is not good name. Just let me know, if you think you get good name for mob.

Magni

Posted on 2006-08-11 15:53:28 UTC.


MOB has a new name. Magni. And what is more important magni has new home.
I created Midgard's subproject on tigris with hope that this kind of application one may find usefull.

It's not designed to be powerfull all in one application. I just wanted to proof that midgard core became powerfull and it's relatively easy to write midgard library dependant application.

First. Midgard core is not a CMS core, it's just database abstracion layer with authentication mechanism and some built in additional features. Thus , core is not your ideas aware. That's good , because its generic purpose doesn't limit your ideas. So together with other components ,( like midgard-apache module or midgard-php ) it turns into powerfull CMS framework which can be written in any language , like MidCOM is written in PHP.

Second. Just imagine that your favourive PHP ( only PHP ) based CMS which has been developed for last three years begins to be interesting for java community. Imagine that CMS contains 50 000 lines of code. 20 000 is database releated code, another 20 000 creates logic and only 10 000 defines user interface. What java community should do in such case? Write 50 000 lines of java code and be in sync with PHP based development branch. Sounds good? No.
Now imagine that you want to build Your new project with python and Midgard as basement. It's enough then to write python language bindings ( which is done only once ) and focus on Your application. Amount of code which must be written depends on project only. You do not have to worry about anything else.

As a good example, midgard-php bindings part for midgard_collector type:

Execute method:

midgard-core C 

MidgardCollector *object;



midgard_collector_execute(object); 



/* SQL query is executed and you do not have to worry about it,



 even if it's C */

PHP binding Zend ( 1 and 2 ) 

PHP_FUNCTION(_execute)



{



/* parse parameters and other Zend relaed stuff */



/* ... */ 


/* Get underlying GObject */



MidgardCollector *object =



(MidgardCollector *) php_midgard_fetch_object(getThis());


/* Invoke method */



midgard_collector_execute(object);



}

And finally midgard-php: 

$mc = new midgard_collector(mixed params);



$mc->execute();

The same you can do with java, pyhon, ruby or perl.
I skipped error handling in examples because it's not a point here.

Third. How many database related webDAV services do you know? 5? 10? Maybe 1? With new midgard core it's possible to write native Gnome VFS module and share Midgard content as it could be mounted cd-rom or floppy. How many Gnome or Nautilus plugins have you seen written in PHP?

Another one thing is: if Your favourite CMS is written in PHP and you need for example python small application for it, which could be running on Nokia 770, you must read the second point of this blog. With new midgard core you depend on language bindings only , it doesn't matter if you need KDE or Gnome desktop app. You doesn't have to know if you are going to use MySQL , MS SQL or PosgreSQL. You do not have to know what SQL is. All you have to know is the language you want to write Your app in.

And at the end , imagine this ( it's only theory , but I bet you get the point ):
Your new website is online shop. And what you need is to know when new item has been ordered via website. You do not have POP mail account , so checking every single minute if order is already there is just crazy and without any sense. So what you exactly need is a tray small application which every minute looks for new orders made via website ( exactly like POP mail tray apps ).
Can you do this if Your powerfull CMS is written in PHP? No.
Can you do this with centralized written in C library ( database abstraction layer ) and decentralized applications? Yes , you can.

And ah! Last but not least! How many code do you need to write to create new classes and database "connectivity" for them using PHP or Python? Plenty. How many code must be written with Midgard? None.

Forgive me if that blog entry is too shameless. It's just true.

Exporting MgdSchema objects

Posted on 2006-08-28 11:36:09 UTC.


Midgard CVS HEAD supports nice and quite usefull export method for MgdSchema objects. It is quite simple to export objects on PHP level.

A code example for exporting whole sitegroup ( SG0 in this case ): 

foreach ($_MIDGARD['schema']['types'] as $type => $val){

        $replication_data_dir = "/some/path";
        $qb = new midgard_query_builder($type);
        $qb->add_constraint("guid", "<>", " ");
        $ret = $qb->execute();

        foreach ($ret as $object){

          $filepath = $replication_data_dir."/".$type."_".$object->guid.".xml";
          $dump_file = fopen($filepath, 'w');
          $xml = $object->export();              

          if($xml) {
                 fwrite($dump_file, $xml);
                 fclose($dump_file);
          } else {
                 echo mgd_errstr();
          }
    }
 unset($object);
}

If you try to execute this code twice , you should get MGD_ERR_OBJECT_EXPORTED errno returned for every export method call.

Coming back from vacation

Posted on 2006-08-29 12:07:33 UTC.


Isn't that quite abnormal? I came back from vacation last sunday and first thing I done yesterday was writting about Midgard objects exporting ;)

So , we spent nice week travelling a bit through Świętokrzyskie Mountains.
"A bit" means that on the back seat of your car is 4 years old girl and never stops talking and always wants to go just there where you do not plan to go :)

Few places of interest: Krzyżtopór castle( who's from Sweden and destroyed it? ;), "Jurrasic park"( this is interesting, Ola was very bored there "Are we going to castle now?" ;), Paradise Cave ( unfortunatelly , no one is able to make pictures there ), Piekoszewice ( castle among houses ).

And other interesting things: my daughter is interested in new Apache Foundation logo, completely new spider-admin , and even dead boar is happy.

In other news: Today, I will implement import functionality for Midgard objects, so basic core based replication should be ready tomorrow.

Midgard2 replication preview

Posted on 2006-09-02 12:43:39 UTC.


Few days ago I wrote about exporting objects from Midgard database. Now it's time to import them to other database or even the same. I can not elaborate about API method, cause it's simply and easy.

bool mgd_import_object(string xml_content);

or static method ( a bit unlogic if you want to tie to particular class ):

midgard_classname::import(string xml_content);

or midgard-core's API:

 gboolean midgard_import_object(MidgardConnection *mgd, gchar *xml);

Usage is very simple , you just pass xml content ( as serialized object's data ) as function or method parameter. Xml content is parsed in memory and particular object instance is created. Then , import method starts to figure out what's the best method which should be invoked:

Object's record identified by guid is not found:

  • if it's already purged , do nothing
  • if it's not found in repligard's table , create new record

Midgard API is used to create new record , so for example you do not have to worry about duplicated name&up pairs.

Object's record is found:

  • if imported object's revised datetime is less recent or the same as the same value of database object return with error number MGD_ERR_OBJECT_IMPORTED
  • in other case , if imported object is marked as deleted , delete database object
  • if both cases do not match update it


So , poor example could look like this: 

$filename = "/path/to/midgard_article_b531b35cf54111daa870f9b1239adb8fdb8f.xml";

$filehandle = fopen($filename, "r");

$content = fread($filehandle, filesize($filename));

midgard_article::import($content);

/* or mgd_import_object($content); */


It's very initial real Midgard2 replication feature, but it perfectly feets a future "big picture" ideas:

  • Send me this document, report. I will reuse it's content as I need.
  • Send me this news with attachment's , I will import it to my MidCOM's app and reuse with self defined parameters.
  • Make fast backup of my objects
  • Send me your last note to my jabber account ( if I am not available , it will be waiting for me)
  • Export only approved documents.
  • Import every single object I got with POST request data.
  • etc etc etc

Past years proved that replication app is always far from being perfect and what you really need is not always doable or is very complex. I hope we're going to implement powerfull and flexible replication feature ever was made for Midgard.

Generating docs with Doxygen

Posted on 2006-09-12 12:53:06 UTC.


Midgard-core documentation is generated with doxygen package. Last sunday I experimented a bit with creating MidCOM docs with it. I was wondering if doxygen is able to generate PHP documentation while it's optimized for C, C++ and Java.

First. Doxygen package includes nice wizard gui invoked with doxywizard command. So when it's running you can simply define what you really need.

  • optimize for particular language
  • create xml output , manual pages or even chm file
  • create graphical diagrams for classes and relations
  • create html ot latex
  • create html in frames or not
  • plenty more features

Second. It was big surprise to see that basic documentation for MidCOM was created in 6 minutes.

Third. It was huge surprise to see that full documentaion with graphical diagrams and source code references was created in 15 minutes.

An interesting thing is the fact that config file created by doxywizard can be easily reused for documentation generated from cron. So in such case all you have to worry about is calling doxygen with specific doxywizard configuration files.

An issue which was clearly noticable was lack of midcom application class. However midcom package has been included in documentation. Probably it's some doxygen issue or simply the fact that MidCOM docs has been created for PHPDoc , so the syntax might be a bit different for both generators. Anyway , first impression was very nice. Doxygen created very nice html pages with classes , variables, modules , diagrams, etc etc. For me personally, for the first time MidCOM docs became readable. Html pages created with PHPDoc seems to be very nice as some IDE plugin with code completion support. Not for reading anyway.

Midgard database indexes

Posted on 2006-09-22 14:00:39 UTC.


A week ago Rambo wrote about MySQL optimization tricks. Today I decided to make detailed tests with database performance.

Idea

I use midgard_query_builder methods to produce SQL query, but the main point is to test database itself so all queries are executed in MySQL prompt as setting properties from MySQL resources is a bottleneck here. 

$qb = new midgardquerybuilder("midgard_topic");

$qb->add_constraint('up', '<>', 0);

$qb->add_constraint('parameter.domain', '<>', '');

$qb->add_constraint('parameter.name', '<>', '');

$qb->add_constraint('metadata.created', '>', '2002-02-01 00:00:00');

$qb->add_constraint('metadata.published', '>', '2002-02-01 00:00:00');

Query:

SELECT topic.guid, topic.sitegroup, topic.metadata_creator, NULLIF(topic.metadata_created,'0000-00-00 00:00:00') AS metadata_created, topic.metadata_revisor, NULLIF(topic.metadata_revised,'0000-00-00 00:00:00') AS metadata_revised, topic.metadata_revision, topic.metadata_locker, NULLIF(topic.metadata_locked,'0000-00-00 00:00:00') AS metadata_locked, topic.metadata_approver, NULLIF(topic.metadata_approved,'0000-00-00 00:00:00') AS metadata_approved, topic.metadata_authors, topic.metadata_owner, NULLIF(topic.metadata_schedule_start,'0000-00-00 00:00:00') AS metadata_schedule_start, NULLIF(topic.metadata_schedule_end,'0000-00-00 00:00:00') AS metadata_schedule_end, topic.metadata_hidden, topic.metadata_nav_noentry, topic.metadata_size, topic.metadata_published, NULLIF(topic.metadata_exported,'0000-00-00 00:00:00') AS metadata_exported, NULLIF(topic.metadata_imported,'0000-00-00 00:00:00') AS metadata_imported,topic.metadata_deleted, topic.metadata_score, topic.up AS up,topic.id AS id, NULLIF(topic.created,'0000-00-00 00:00:00') AS created, topic.revisor AS revisor, topic.score AS score, topic.code AS code, topic.creator AS creator, NULLIF(topic.revised,'0000-00-00 00:00:00') AS revised, topic.name AS name, topic.revision AS revision, topic.owner AS owner, topic_i.title AS title, topic_i.lang AS lang, topic_i.description AS description, topic_i.sid AS sid, topic_i.extra AS extra FROM topic_i,topic,record_extension WHERE (topic.up <> 0 AND record_extension.domain <> '' AND record_extension.name <> '' AND topic.metadata_created > '2002-02-01 00:00:00' AND topic.metadata_published > '2002-02-01 00:00:00' AND topic.name <> '') AND topic.id=topic_i.sid AND topic_i.lang IN (0, 0) AND topic.guid = record_extension.parent_guid AND topic.guid = record_extension.parent_guid AND topic.sitegroup IN (0, 0) AND topic.metadata_deleted = FALSE;

Note that we use SQL standardized NULLIF control flow function. So on PHP level , never write code like : 

if ( $object->metadata->created == '0000-00-00 00:00:00')

but instead write it simpler and faster: 

if ( $object->metadata->created)

The test

MySQL Cache is off.
Query is executed 3 times every time when table is altered and I use average time.

Indexes on id and sitegroup.

* 13 rows in set (18.96 sec)

CREATE INDEX deleted_idx ON topic (metadata_deleted);

* 13 rows in set (19.00 sec)

CREATE INDEX name_idx ON topic (name);

* 13 rows in set (19.04 sec)

CREATE INDEX up_idx ON topic (up);

* 13 rows in set (19.06 sec)

CREATE INDEX metadata_created_idx ON topic (metadata_created);

* 13 rows in set (19.05 sec)

CREATE INDEX metadata_published_idx ON topic (metadata_published);

* 13 rows in set (19.12 sec)

CREATE INDEX record_extension_oid_idx ON record_extension (oid);

* 13 rows in set (19.04 sec)

CREATE INDEX record_extension_domain_idx on record_extension (domain(255));

* 13 rows in set (19.04 sec)

CREATE INDEX value_idx ON record_extension (value(255));

* 13 rows in set (19.05 sec)

CREATE INDEX name_idx ON record_extension (name(255));

* 13 rows in set (19.02 sec)

CREATE INDEX parent_guid_idx ON record_extension (parent_guid(80));

* 13 rows in set (0.01 sec) (!!)

CREATE INDEX tablename_idx ON record_extension (tablename(255));

* 13 rows in set (0.01 sec)

All indexes dropped ( except id and sitegroup ).

* 13 rows in set (19.25 sec)

Similiar query , but without parameters. 

$qb = new midgardquerybuilder("midgard_topic");

$qb->add_constraint('up', '<>', 0);

$qb->add_constraint('metadata.created', '>', '2002-02-01 00:00:00');

$qb->add_constraint('metadata.published', '>', '2002-02-01 00:00:00');

I executed this query with and without metadata indexes. The difference between queries time was none.

Conclusion

Indexes for referenced properties ( on object level and "foreign keys" on database level ) are the keys for performance. And indexes used for queries when the same table is used seem to not increase performance at all.
However this can be different for queries which join the same table.



midgard-project.org update

Posted on 2006-10-06 19:01:07 UTC.


Today I updated Midgard setup on the host which serves www.midgard-project.org website. Unfortunatelly I couldn't use datagard as remotes database server doesn't allow login in as database administrator, so update has been made step by step manually.
A good news is the fact that debian packages was used to update Midgard environment.

If you need to update remote database and can not use datagard , try this script:

Database update 

#! /bin/sh

. /usr/share/midgard/datagard/datagard.conf

# Set default initial values

_DATABASE_NAME="midgard"

_DATABASE_HOST="127.0.0.1"

_MGD_DB_USER="midgard"

_MGD_DB_PASS="midgard"

_SG_USER="admin"

_SG_PASS="password"



mgd_set_config_data MGD_DB_ACTION Update

mgd_set_config_data MYSQL_ADMIN_USER root

mgd_set_config_data MYSQL_ADMIN_PASS ""

mgd_set_config_data MGD_DB_HOST $_DATABASE_HOST

mgd_set_config_data MGD_DB_USER_NAME $_MGD_DB_USER

mgd_set_config_data MGD_DB_USER_PASS $_MGD_DB_PASS

mgd_set_config_data MGD_DB_NAME $_DATABASE_NAME

mgd_set_config_data MGD_SG_USER_NAME $_SG_USER

mgd_set_config_data MGD_SG_USER_PASS $_SG_PASS

mgd_set_config_data MGD_UPDATE_DB $_DATABASE_NAME

mgd_set_config_data MGD_DB_CONNECT_HOST $_DATABASE_HOST



mysqlcmd="mysql  --host=$_DATABASE_HOST --user=$_MGD_DB_USER --password=$_MGD_DB_PASS"



mgd_update_db

mgd_update_db_for_utf


This is of course example , and you might want to tweak it for particular needs.

 A few nice features I discovered (oops , I wrote it :) after updating midgard:

  • All SQL errors are reported to apache error log file whithout any need to change apache log level and without any need to reboot. Took me long while to find apache main error log , but at last I was nicely surprised.
  • Schema validation is much much more better than this one wich exists in 1.7 branch. 
Starting apache 1.3 web server...midgard-core (pid:30299):(WARNING):
Wrong node name 'propetty' 
in '/usr/share/midgard/schema/org_openpsa_core.xml' on line 137

B&W

Posted on 2006-10-10 22:28:18 UTC.


It's not actually Black and White. It's almost black with some minimal amount of white ( or grey ).

Anyway , I like it :)

Does anyone from Midgard community has something with colours?

Set of scripts

Posted on 2006-10-18 16:48:22 UTC.


While googling yestarday, I found few old entries about Midgard on Polish mailing lists. A few of them:

  • Midgard is only set of scripts
  • Midgard is distributed only as selected distros' binaries

Isn't it funny? :)

I am looking forward for multilanged MidCOM so I think some Midgard documentation or concpets could be easily translated to Polish then.

Another interesting thing is the fact that I almost finished new parameter method for Midgard objects.
A new one is ( tries to be ) much clever then legacy one and uses parameters' cache based on midgard_collector.

Simply it means that first call for some parameter in some domain , forces core to cache all domain parameters, so later call to the same parameter or another one in the same domain is "as slow as" : 

$value = $arry['key']

No database selects.

This implementation will be introduced in Midgard since 1.8.1.

And something for high-tech fans: Midgard 1.9 is going to be released soon!

Midgard 1.9 work just started

Posted on 2006-10-24 19:23:45 UTC.


Bergie just mentioned about Midgard 1.8 and first cleanup commit.
As a result of today's work we have partially legacy free and compilable Midgard framework. Partially means that I am not so crazy to remove everything at once, as 1.9 release might use some legacy code or solutions to provide smooth CVS and alpha releases installations.
There is no repligard, no Aegir, no Spider. No legacy API in midgard-php extension. No Apache1 module.

Next few days I will implement ligbda support , so 1.9.0alpha1 release is almost here.

I realized today, how much work is still left to get fully Midgard2...

And quoting Arttu:

Midgard is an IT project so we can expect delays. Seven years sounds rather long, but Rome wasn't built in a day and conquerring the world is even more difficult.

During last two years we made almost 20 releases. And we brought Midgard to desktop. We just made very unique and powerfull CMS/Framework system. We even can not compare Midgard with other CMS'es. Simply , they can not be used in Apache and desktop environment.
Maybe Midgard2 is postponed seven years. But last two years really count!

Midgard 1.8 - PEAR issues on debian stable

Posted on 2006-10-27 12:21:18 UTC.


There is an issue with PEAR on debian stable. So Installing Midgard 1.8 is a bit difficult. As a workaround I found this solution a good start to install all PEAR packages. 


# Upgrade PEAR on debian stable

pear upgrade Archive_Tar

pear upgrade pear-1.3.3 && pear upgrade pear



# Discover channels



pear channel-discover pear.midcom-project.org

pear channel-discover pearified.com





pear install pearified/Role_Web
pear install midcom/Role_Mgdschema

#Debian related stuff 
pear config-set mgdschema_dir /usr/share/midgard/schema
pear install midcom/Role_Midgardsql
pear config-set midgardsql_dir /usr/share/midgard/sql/update
pear install midcom/Role_Midgardelement 

# By default this is midgard, but you might want to change it
pear config-set midgard_config_file /etc/midgard/conf.d/midgard




# Install packages
pear install pearified/JavaScript_Prototype

pear install Mail_Mime

pear install midcom/midcom_helper_datamanager2

pear install -f midcom/org_openpsa_httplib

pear install -f midcom/net_nemein_calendar

pear install -f midcom/midcom_helper_imagepopup

pear install -f midcom/midcom_helper_search

pear install -f midcom/de_linkm_sitemap

pear install -f midcom/midgard_admin_sitewizard

pear install -f midcom/net_nehmer_static

pear install -f midcom/fi_protie_navigation

pear install -f midcom/Javascript_protoToolkit

pear install -f midcom/midcom_helper_datamanager

pear install -f midcom/midcom_admin_content

pear install -f midcom/no_bergfald_rcs

pear install -f midcom/org_routamc_photostream

pear install -f midcom/org_routamc_gallery

pear install midcom/net_nehmer_blog


After running all these commands I suggest to re run datagard and select PEAR again from menu. Sitewizard will be set correctly then , even if PEAR packages will fail to install from datagard menu.

Updated: I removed pear install -f midcom/net.siriux.photos

Updated:

  • Removed pear install -f midcom/midcom_helper_toolbar
  • Added debian related instructions

Updated (16.11.2006): Discover channels before installing packages.
( Excellent point from Sentrum2 , #midgard :)

Midgard and PostgreSQL, first issues

Posted on 2006-10-28 20:56:18 UTC.


Just had to copy it here. Instead of screenshot.

This is well know play. Find three words which doesn't exist in Midgard documentation.

Schema (pid:7822):(m): Type midgard_language
midgard-core (pid:7822):(WARNING): Midgard connection error: ERROR: syntax error at or near "SET" at character 30, No:(-1), src:gda-postgres

SQL: Not available
midgard-core (pid:7822):(debug): query = INSERT INTO midgard_language SET locale='', code='', description='', name='_schema_test_create-1392903333', native='', guid='64f93ec466ab11dba55db529ad5a9fa89fa8',sitegroup=0,metadata_creator='', metadata_created='2006-10-28 17:40:25+0000', metadata_revised='2006-10-28 17:40:25+0000', metadata_revision=0, metadata_revisor='', metadata_size=115 ,metadata_locker='' ,metadata_locked='' ,metadata_approver='' ,metadata_approved='' ,metadata_authors='' ,metadata_owner='' ,metadata_schedule_start='' ,metadata_schedule_end='' ,metadata_hidden=0 ,metadata_nav_noentry=0 ,metadata_published='2006-10-28 17:40:25+0000'

** (MIDGARD:7822): CRITICAL **: gda_postgres_provider_get_last_insert_id: assertion `GDA_IS_POSTGRES_RECORDSET (recset)' failed


Why Plone needs Midgard

Posted on 2006-11-01 00:32:19 UTC.


Recently , I followed these threads:

I wonder how Zope and Plone can be extended?

Can I use Java with Plone?
Can I use PHP with Plone?
Can I use pure GTK+ with Plone?
Can I use SQLite or PostgreSQL or MySQL with Plone?

I wonder about such sentence:

If you can see the beauty, then Python got you

And I wonder about comprehensive and standardized tools which can be reused in any environment.

Zope and Plone developers, Midgard uses GLib and GObjects as its base. If you need to extend Zope , just use Python language bindings for Midgard. It's easy.

I can of course implement PHP bindings for Plone.


ZEND2 - Developer's hell

Posted on 2006-11-27 13:35:48 UTC.


Recently I had to fix few important issues related to Zend2 engine and midgard-php extension. All issues were relatively easy to fix , but I realized that hell is coming.

some_class::static_method();

Such language construction works fine with PHP4 and with PHP5.
The difference is that with Zend2 ( PHP5 ) 'static_method' must be explicitly defined as zend static method and not as function ( PHP4 ).
Not a big deal, but ZEND_METHOD macro is expecting class name for which static method is implemented. And this simply can not be passed to macro as variable and must be hardcoded. Well, I could be a writer if I would like to write , write and write. I must reserve endless hours to implement midgard-php extension as pure PHP5 one. Hacks, trials and errors, segfaults, hacks, trial end errors... looping. I didn't read Zend2 sources yet and have no idea how ZEND_METHOD is implemented but I expect Zend guys still use sizeof instead of strlen in such macros so what absolutely must be done is to forget about Zend2 API and write own solution when you work with unknown variables.

Another interesting issue is the fact how GObjects ( or GTK ) ones can be implemented for PHP. For PHP5 I experimented with direct pointers arrays, so there were no need to add some magic __res property to PHP5 objects.
Unfortunatelly it fails on bigger loops like:

while(something) {
$var = new some_class();
}

The solution is to only use __res property.

I wonder how it's safe if one can accidentally or explicitly get "forbidden" object by changing object's __res property value. This is a scripting language so fivth object instance will always have __res value = 5. If this is some secret account for example I can easily change my object's __res value to 5 and get object which shouldn't be available for me.

And another interesting thing is memory usage and kind of "optimization".
I almost finished basic midgard_replicator implementation. And mandatory strings duplication makes me frustrated for months. Why the hell I can not reuse string contents and I have to duplicate it over and over?
Why string variable initialized without duplication parameter ( that's from docs ) triggers segfault when variable is destroyed by zend?

I hope I am wrong. And if I am and some Zend developer is reading it, please be so kind and give that magicall url where such issues are very well documented.

Midgard-apache2. Refactoring.

Posted on 2006-12-28 12:53:06 UTC.


I already blogged about libgda implemented in new midgard-core.
While it looks promising for command line applications it started to be real problem in Apache2 environment. First of all I began to refactor midgard-apache module with new GObject based API.

As soon as I began I realized something really bad happened:

GLib-GObject-WARNING **: cannot register existing type `midgard_connection'

The midgard-core API call in this case could be simple limited to two lines of code:

g_object_new(MIDGARD_TYPE_CONNECTION, NULL);
gda_client_open_connection_from_string(client, "MySQL", str, 0);

Why such two lines of code can trigger such warning? ( Keep in mind that this "simple warning" was only a start of endless critical errors reported and followed by segfaults ).

I started to check code, recompile midgard-core and midgard-apache with different flags. But none of this things seemed to help. I started to investigate libgda code and found that the source of the problem is GLib's GModule API. Well, rather Apache2 not being compiled with gmodule flags.

One is able to easily reproduce it with such simple code:

g_object_new(GDA_TYPE_CONNECTION, NULL);
g_module_open("/path/to/any/lib/module.so", G_MODULE_BIND_LAZY);

I tried libgda and evolution's plugin in my case. It perfectly breaks GType system.

OK, I decided to try this code not when midgard-apache module is initialized but when http request is made. And it seemed it worked. But not for a long time:

midgard-core (pid:14649):(WARNING): Unable to create the configuration file.
I/O error : Permission denied
I/O error : Permission denied
midgard-core (pid:14649):(WARNING): Error saving config data to /root/.libgda/config

It's not enough I am laying down, I must be kicked....
gda_client_open_connection_from_string in this case is a hack and this function creates configuration file , reads data from this file and when data ie read , file is cleaned. But it's not trivial when Apache's child is not running with root permission...

I started to write example codes to find any workaround quickly, and finally I changed midgard_connection and midgard_config code a bit, so some of API functions can easily be invoked even if no object instance was created. Manipulating with core's structures ( available as public API structures ) I managaed to create database connection using libgda API ( when module is initialized ) and create midgard_connection objects when http request is made.

So far it looks very good. Especially midgard_connection object looks good as its instance is created once and available as one object per virtual server ( Midgard1 module uses one connection structure per request). It looks like a very good start for future caching possibilities.

Back

Layout Copyright © 2006 Finnish Teleservice Center Ltd Oy - Site Powered by Midgard CMS