Is Jabber/XMPP the ultimate IM solution? Imho, not really.

This is a small rant about XMPP protocol 🙂

As a user I see problems with basic stuff like emoticons, encryption, conference – nothing works perfect and out of the box.

Yes… XMPP allows most of things I need.. but they do NOT work OUT OF THE BOX nor on all platforms.

As a developer, XMPP is not that open, and it is horribly complicated.

Jabber (and XMPP) are protocols, that, together with entire “ecosystem” around it (applications that implement this protocol, various add-ons, etc) allowed a nice, open source, alternative in world om IM.

But still – XMPP seems to lack many things. I can not say it satisfies my own needs; and probably as well needs of many other user groups.

What I would expect from full IM solution:

1. open source and ported to other platforms, also embbed/cellphones/etc
2. option to use emoticons but also to disable them for given conversation
3. transfer files
4. option to send ALERT (bell)
5. logging the conversations
6. encryption, also with OpenPGP, as well as with one time password
7. conferences – group chat
8. easy creating, joining group chats
9. all above must work as well in group chat version (except it applies to all members)
10. status, also invisible, description
11. list of users that can be searched
12. list of contacts stored on server
13. easy options to ignore given contacts or group, or hide from them etc
14. storing of all messages on server while offline
15. robust protocol, nothing gets lost even if connection brakes or computer crashes
16. voice chat and conferencing. also with video
17. all of above must JUST WORK, without f-ing with gazilion applications, of which all not fully implement protocol or have own quirks and problems

Jabber seems to fail if we want to use all of the above needs.
Even when discarding audio and video, simple text chat needs a lot more to fullfill my goal.

In addition, as a developer, I would like…

1. to have simple protocol…
2. to not have to send huge amounts of XML to say “hi.”
3. to have nice ready to use implementations that are really usable (free for commerciall use, also in popular languages like say C, C++, including servers that implement all functions etc)

XMPP/Jabber fails imho because:

• the above things “work” .. in theory, or in some applications (where other things do NOT work)
• there is no ONE SINGLE GOOD implementation (also open and portable)
• even there is NO ONE PROTOCOL! Read about XMP extensions… imho this is just chaos.

Overall, I would say that XMPP fail to provide (at least) ONE, good an UNIFORM platfrom, because XMPP is too extensible, while forgetting of need to select important futures and implement them fully and in a polished way.

Bug report and system info tool for ubuntu

When reporting bugs in ubuntu I have to type in each time description of my system (more detailed if bug is likelly to be connected to drivers), some software versions info etc.

I would like to see a tool that would easly generate such information:

• fast and easy
• generates copy/pastable text
• system info
• hardware info
• option to prepare the information with given problem/bug in mind:
• option to give more info on given topic (i.e. on GFX)
• option to give more info on given program
• usable for automatic parsing later
• human readable as well
• extensible

Example (draft)

$ sysdescr
[SysDescr Ubuntu 8.04 amd64; 2.6.24-19-generic; Athlon64 X2; 2 GB ram; Upgraded=20080826]

or

$ sysdescr gfx

[SysDescr Ubuntu 8.04 amd64; 2.6.24-19-generic; Athlon64 X2; 2 GB ram; Upgraded=20080826
ram[2 GB, 30% appl, 3% free]
GFX[
card[Nvidia6800GT, pci:5, 512MB]
xorg[1:1.3.0-6 running pid=5123 OK] DM[kdm]
screen[user1 kde on :0 VT7 HANGED[in “epool(5,…” in backtrace1] PID=3211 ]
[root gnome on :1 VT8] ]

]
backtrace1[
_checkxsomething 0x094343 …..
….
etc
]

adjtime() and synchronizing time in linux kernel

Time problem

My main board clock is really broken – running too slow. Ntpd network time synchronizer tries to fix it, but the drift is so big (10 second per hour) that linux kernel can not adjust the time using adjtime(). Adjtime usually is good, because it applies the time change gradually.

The only thing that works for me is settimeofday() which is not as good because then time “jumps” instead gradually change to correct time, distrubing movies playback and lots of other stuff.

So, I would like adjtime() to be able to adjust the time gradually but FASTER, to work for really broken hardware clocks.

Also, time ntpd program could be smart enought to see that the time difference is really bad, and could start using settimeofday as ntpdate does… I reported this originally as ubuntu bug in ntp.

Details – example

During 2 hours 20 minutes my clock drifted by 20.2 sec; So 0.0024 sec/sec time drift.

During 1 minute adjtime() makes kernel fix the drift by 0.03 sec. So, 0.0005 sec/sec time adjust.

Therefore kernel time adjust is too slow to compensate for time drifting of my hardware clock.

Solution

The correct solution seems to be to add another syscall that tells kernel how fast should the time be adjusted.

In lib C currently we have:
int adjtime(delta, olddelta);
and my idea is:
int adjtime_speed(delta, speed, olddelta, oldspeed)

Then, programs like ntpd would observe the time drift, and issue orders to apply the correction faster.

Also, for clocks that are just slower in a linear/constant way, it would be nice to add:
int time_set_drift(double constant_drift);
time_set_drift(0); // normal clock
time_set_drift(0.001); // clock that always losts 1 second per 1000 seconds
etc.
Then ntp deamon would first observe how system clock reacts to adjust commands, and if a constant drift is discovered and messured, kernel would be told how to fix it.
That way the correction would work even if system is offline and ntpd doesnt work normally.

Any comments are appreciated 🙂  Is this worth implementing? Any hints which kernel files to look at?
Anyone had similar problems?

Cache policy in Linux

Linux caches reads/writes (in cache/buffer) in memory (usually RAM) to improve read times when user again accesses the same data.

The problem is, it seems there are no good tools to say which files we want to cache and which not(*). And since limit of cache is limited, we often end up caching data that will be not used often.

Small example – computer 2 GB ram, that runs normally and caches 1 GB of frequently used files.
Then we do some one time operation like write zero’s to a device to whipe it out (or use shred utility) etc. As a result computer will try to cache as much as possible from 80 GB of zero’s that are written to the device, and NOT caching the things we actually need.

Admin knows that  cat /dev/zero > /dev/something is a one-time operation and no need to cache it, but how to tell that to kernel?

(*) there is madvise syscall, but it seems it can not be applied to other processes (as it is possible with nice/renice). But perhaps it can be used to patch most important tools, i.e. used to backup, move files, etc.

Test case

Well, obviously create and read a file,  then create other huge file (or cat /dev/zero to unused device WARNING: IT WILL BE WIPED OUT!!!) and try to read oryginal file.

1. dd if=/dev/urandom  of=./mydata count=1024 bs=409600
2. time sha1sum ./mydata # will be fast
3. cat /dev/zero > /dev/somedevice # I know I dont need to cache this
4. time sha1sum ./mydata # will be slow

Solution

Develop tools to do something like:

hintmem -nocache cat /dev/zero > /dev/something

It would be like nice (and ionice), but to hint the kernel about max size of cache to use.

Or better yet, develop some “memory pressure” system controll that in more fluid way. It would be like telling “cache things I use often, especially cache all create by process openoffice, and rather not cache things created by this this and that process since its probably one-time”.

IRC groups – solution to battles over channel policy?

The Problem

Many IRC channels vary in they allowed discussion style, in example on freenode,

• ##linux – brave ideas=YES f-word=NO trolling=NO insults=NO
• ##c++ – brave ideas=YES f-word=YES trolling=quite insults=rather not
• ##c++-social – brave ideas=YES f-word=YES trolling=YES insults=quite
• #politics – brave ideas=YES f-word=YES trolling=YES insults=YES
• ##ubuntu – brave ideas=NO f-word=NO trolling=NO insults=NO anything not candy nice PG rated will get a ban

This leads to people complain about “bad ops” etc and to lots of frustration.

Such censorship may be in fact needed – It is users freedom to say “fuck“, but then again it’s channel owners/creators freedom to say they don’t want this on their channel.

On #politics the solution is that no one gets banned, but everyone uses /ignore often to “customize” what they want to see.

This is however troublesome for typical channels.
When new users come to #ubuntu for quick help – they really want to avoid any flaming, foul language, and they often do not know how to use /ignore so it’s their freedom to NOT see such messages.

Other typical – but not very usable – “solution” is to “fork” the channel, that is create alternative one, say ##linux-uncensored instead ##linux etc. This idea often fails, because the new channel have 1-5 users invited by hand, while the main channel have obvious name, and often IRC clients are preconfigured to join it (i.e. irc client in Ubuntu join #ubuntu etc).

Right to say and right to not listen – solution

My solution aiming to fix that – divide channel into groups. Each user can unsubscribe groups he doesn’t like. And operators do not kick people, but limit them to only given group.

Example, lets take some channel, say #sdl – with around 100 users it doesn’t have good chance to be divided into #sdl-offtopic and such – because channels will be too small probably. However still the users may disagree what each want to see. So, lets introduce groups in this channel:

• #sdl – the main channel, behave nice, talk on topic, etc
• #sdl.idea – fighting and a bit of “trolling” about ontopic subjects
• #sdl.fword – strong language allowed
• #sdl.offtopic – social random chat allowed
• #sdl.anarchy – everything allowed

Each user choose which groups he will “hear” (by default: most groups, but he can ignore say offtopic if he is bussy)
and as which group he will “speak” (by default: all groups, but he can “go to” .fword if he is about to say what he thinks about this damn &*#&#@)

So, how it will work?

When you join #sdl you subscribe groups: main, idea, fword, offtopic (but NOT to anarchy) by default – so You hear all the talk.
If someone is making troubles in the channel, he is moved to .anarchy, and then You do not hear him (unless you subscribe .anarchy)
If You want to do some random chat, then you go to group .offtopic ; Then all what You say will be marked as offtopic – so that people who ignored .offtopic will not be bothered.
If You curse a lot, then instead being kick-banned, operator just moves you to .fword.

As a result, each listener chooses what style of conversation he wants to hear and what not,
and for speaking – you can choose which rules you want to follow.

Also, server may automatically make sentences with “fuck” to be sent as from .fword and/or move You there.