Planet GRPUG

May 16, 2012

Whitemice Consulting

Informix Dialect With CASE Derived Polymorphism

I ran into an interesting issue when using SQLAlchemy 0.7.7 with the Informix dialect.  In a rather ugly database (which dates back to the late 1980s) there is a table called "xrefr" that contains two types of records: "supersede" and "cross".  What those signify doesn't really matter for this issue so I'll skip any further explanation.  But the really twisted part is that while a single field distinquishes between these two record types - it does not do so based on a consistent value.  If the value of this field is "S" then the record is a "supersede", any other value (including NULL) means it is a "cross".  This makes creating a polymorphic presentation of this schema a bit more complicated.  But have no fear, SQLAlchemy is here!

When faced with a similar issue in the past, on top of PostgreSQL, I've created polymorphic presentations using CASE clauses. But when I tried to do this using the Informix dialect the generated queries failed. They raised the dreaded -201 "Syntax error or access violation" message. 
The Informix SQLCODE -201 is in the running for "Most useless error message ever!".  Currently it is tied with PHP's "Stack Frame 0" message.  Microsoft's "File not found" [no filename specified] is no longer in the running as she is being held at the Hague to face war crimes charges.
Rant#1: Why do developers get away with such lazy error messages?
The original [failing] code that I tried looked something like this:
class XrefrRecord(Base):
    __tablename__  = 'xrefr'
    record_id      = Column("xr_serial_no", Integer, primary_key=True)
    ....
    _supersede     = Column("xr_supersede", String(1))
    is_supersede   = column_property( case( [ ( _supersede == 'S', 1, ), ],
                                            else_ = 0 ) )
 
    __mapper_args__ = { 'polymorphic_on': is_supersede }   
   
   
class Cross(XrefrRecord): 
    __mapper_args__ = {'polymorphic_identity': 0} 
           
           
class Supsersede(XrefrRecord): 
    __mapper_args__ = {'polymorphic_identity': 1}
Text#1: Code snippet that does not work.
The generated query looked like:
  SELECT xrefr.xr_serial_no AS xrefr_xr_serial_no,
         .....
         CASE
           WHEN (xrefr.xr_supersede = :1) THEN :2 ELSE :3
           END AS anon_1
  FROM xrefr
  WHERE xrefr.xr_oem_code = :4 AND
        xrefr.xr_vend_code = :5 AND
        CASE
          WHEN (xrefr.xr_supersede = :6) THEN :7
          ELSE :8
         END IN (:9) <--- ('S', 1, 0, '35X', 'A78', 'S', 1, 0, 0)
Text#2: Query SQLAlchemy generates for that construct.
It would seem that this would work.  If you substitute the values for their place holders in an application like DbVisualizer - it works.

The condition raising the -201 error is the use of place holders in a CASE WHEN structure within the projection clause of the query statement; the DBAPI module / Informix Engine does not [or can not] infer the type [cast] of the values.  The SQL cannot be executed unless the values are bound to a type.  Why this results in a -201 and not a more specific data-type related error... that is beyond my pay-grade.
Notice that when used like this in the projection clause the values to be bound are both input and output values.
Aside#1: An existential dilemma
The trick to get this to work is to explicitly declare the types of the values when constructing the case statement for the polymorphic mapper. This can be accomplished using the literal_column expression.
from sqlalchemy import literal_column

class XrefrRecord(Base):
    _supersede    = Column("xr_supersede", String(1))
    is_supersede  = column_property( case( [ ( _supersede == 'S', literal_column('1', Integer) ) ],
                                               else_ = literal_column('0', Integer) ) )
 
    __mapper_args__     = { 'polymorphic_on': is_supersede }
Text#3: A working CASE based polymorphic declaration.
Visually if you log or echo the statements they will not appear to be any different than before; but SQLAlchemy is now binding the values to a type when handing the query off to the DBAPI informixdb module. 

Happy polymorphing!

by whitemice (noreply@blogger.com) at May 16, 2012 02:30 PM

May 09, 2012

Ben Rousch's Cluster of Bleep

Ubuntu 12.04 Splash Screen Lockup with LiveCD

I just setup Kubuntu Linux 12.04 on a HP Pavilion zv6000 and ran into a pesky bug. This bug is on all of the Ubuntu and Kubuntu 12.04 final release CDs and ISOs, so this is going to be a common issue for a while. In this post I’ll show you how to work around the dreaded splash screen lockup due to b43 missing firmware bug.

The first symptom of the problem is that your computer locks up on the splash screen while booting the Ubuntu or Kubuntu 12.04 LiveCD. This can also happen during the first boot after upgrading from Ubuntu 11.10 to 12.04 if you haven’t previously installed the Broadcom drivers in 11.10. At this point you can’t see the error, all you see is a stalled splash screen. You will have to long-press the power button on your laptop to shut down.

If you’d like to see the error, reboot from the LiveCD, and select F6 at the Install/Memtest/Check CD for Defects screen. Then delete the “quiet” and “splash” from the boot command line and continue booting from the CD. Eventually you’ll see:
[ 95.514833] b43-phy0 ERROR: Firmware file "b43/ucode5.fw" not found
[ 95.514847] b43-phy0 ERROR: Firmware file "b43-open/ucode5.fw" not found
[ 95.514855] b43-phy0 ERROR: You must go to http://wireless.kernel.org/en/users/Drivers/b43#devicefirmware and download the correct firmware for this driver version. Please carefully read all the instruction on this website.

Again, at this point you will have to shut down by long-pressing the power button.

To get around the problem and actually install Ubuntu, you’ll have to boot from the LiveCD as before, press F6 at the Install/Memtest/Check CD screen, and add the following to the boot options: b43.blacklist=yes . Then when you continue booting, you’ll be able to login and install.

However you will lock up again after the first reboot due to the same error (this time in your installed Ubuntu instead of the LiveCD). You get around the problem in a similar manner. When the GRUB screen comes up, press ‘e’ to edit the boot options for the first boot entry. Add the b43.blacklist=yes to the boot options line, and then continue booting.

Once you’ve successfully booted in Ubuntu or Kubuntu, make sure you have a wired Internet connection and issue these commands at the terminal or konsole to install the Broadcom Wifi firmware: sudo apt-get install firmware-b43-installer . Reboot, and everything should work, including your Wifi.

Here are a few of the sources I used to gather this information:

by brousch at May 09, 2012 08:56 PM

May 04, 2012

Ben Rousch's Cluster of Bleep

Video Editing on Linux at The Open Source Futures

I gave an hour and a half demo of video editing on Linux at the May Open Source Futures meeting in Lansing. I covered a little bit of FFMPEG and a lot of the OpenShot non-linear video editor. There were a lot of questions at the meeting, so I put together this blog post full of useful commands and links to help answer a few of the questions I couldn’t answer at the meeting.

Preparing Your Ubuntu-based Linux for Video Editing
There are a gazillion codecs and video editors and players for Linux. Here are some of the things I always use to prep my machine for video editing joy.

The Medibuntu Repository (for great codecs!)
You want to make sure you have all of the video and audio codecs you will need before you start. The Medibuntu repository is the best place to get them. Just run the script on the Medibuntu page to install the repo, and then run the following commands to install all of the good stuff. These commands assume you’re running a Linux distro based off of Ubuntu 11.10 or 12.04.

  • FFMPEG and gobs of codecs: sudo apt-get install ffmpeg libavcodec-extra-53 libavutil-extra-51 w64codecs libdvdcss2 h264enc libquicktime2
  • GUI Players and Utilities: sudo apt-get install vlc avidemux
  • OpenShot Video Editor: sudo apt-get install openshot blender inkscape
  • Everything You’ll Need: sudo apt-get install ffmpeg libavcodec-extra-53 libavutil-extra-51 w64codecs libdvdcss2 h264enc libquicktime2 vlc avidemux openshot blender inkscape
  • Encode a video for iPhone or iPod Touch

You can also use OpenShot via a LiveDVD. I haven’t personally tried this method.

Useful FFMPEG Commands

  • Convert (almost) any video file to an editor-friendly format: ffmpeg -i in.mp4 -vcodec mpeg4 -sameq -r 30 -acodec libfaac out2.mp4
  • Convert a video to a series of JPEGs: ffmpeg -i in.mp4 -vcodec mjpeg -sameq temp/out-%d.jpg
  • Fix some out-of-sync audio: ffmpeg -i infile -r 30 -sameq outfile

OpenShot Info

Python and Django Groups “Near” Lansing
The subject of Python and Django groups in the area came up, so I put together this quick list.

If there are any more questions, especially any that I missed from the meeting, please ask them in the comments below.

by brousch at May 04, 2012 10:42 PM

May 02, 2012

Ben Rousch's Cluster of Bleep

Responsive Design at MoMoGR

MoMoGR Web Meets Mobile PanelAt their last meeting, the Mobile Monday Grand Rapids (MoMoGR) group asked me to fill in for a speaker and panelist who had to cancel. The topic of the night was “Web Meets Mobile” so the responsive design talk I’d given at the Grand Rapids Web Development Group (GRWebDev) meeting the week before was a perfect fit. I revisited the talk over the weekend and made some changes to the slides to fit the shorter time limit and static format (slides only, no live action). Although I think the new version of the presentation was less impressive live, the slides from it are better for viewing afterwards. Here are the revised slides along with my notes, which are basically my whole script for the presentation: My Responsive Design Slides and Notes Rev. 3

Ben Rousch Presenting at MoMoGR Web Meets MobileAdaptive Images

One issue I mention in the talk which I hadn’t seen a solution to is adaptive image sizing. You use big images to make the big version of your site look good, but then you end up sending the same big images to mobile devices, which wastes bandwidth and mobile processing power while it downsizes the images on the device. I came across these two articles on the WebMonkey website today which address the problem. They both require Javascript, which is kind of a bummer, but beggars can’t be choosers, so here you go: Build Faster Mobile Websites With ‘Adaptive Images’ and JavaScript Package Offers a Smarter Way to Serve Hi-Res Images

Adaptive Menus

An issue which I haven’t touched on in the talks is adaptive navigation and menus. I came upon an article from The Filament Group which discusses some problems and strategies for dealing with these elements in responsive designs: A Responsive Design Approach for Navigation, Part 1

by brousch at May 02, 2012 06:07 PM

April 24, 2012

Ben Rousch's Cluster of Bleep

My Responsive Design Presentation at the April 2012 GRWebDev

It took me about a year and a half, but I finally gave a presentation at the Grand Rapids Web Development Group meeting, An Ode to a One Page Website”. It went well and was a lot of fun. I spoke about responsive design using The Semantic Grid System as applied to a one page website (the over-arching topic of the meeting). About 45 people were in attendance.

Here are the slides from my responsive design presentation (including notes).

Here is a copy of the Dr. I website as used during my presentation.

And finally here is a link to the viewport article I mentioned during the presentation.

I’ll link to the video once when it becomes available.

by brousch at April 24, 2012 02:10 PM

April 18, 2012

Ben Rousch's Cluster of Bleep

A Free Viewer for Microsoft Outlook .msg Files

In my day job I ran into a situation where one of my users needed to read the contents of a bunch of .msg files. We use Google Apps and don’t have Microsoft Outlook installed anywhere, so this was a bit of a problem. I hate slogging into the seedy world of Microsoft shareware, but after a quick search of my Linux tools that’s where I ended up.

To make a long story short, I eventually found a free cross-platform viewer for these Microsoft Outlook .msg files so I thought I’d share it with the Internet. Hopefully someone will stumble on this post and won’t have to download crippled trialware or risk viral infections from seedy software sites.

The aptly-named MSGViewer is what worked for me. It’s a Java webstart application, so it’s cross-platform. However it did not run on Java7; I had to downgrade to Java6 to get it to work. It is lacking print functionality, but you can copy and paste the email contents to something else if you need to print it.

by brousch at April 18, 2012 03:16 PM

Four Forty-Five This Morning

I was making out with my wife in our bunk in a mining colony on an alien world run by our teleporting trans-dimensional sparkly vampire overlords who prefer to travel by miniature steam train when I woke up to George’s softly padding feet and dragging Ledo (blanket) as he plodded into our bedroom. It was 4:45AM.

George: Momma, Daddy, can I have a Yoshi?
Me: It’s very early, go back to bed.
George: But can I have a Yoshi?

I could tell this wasn’t going to end soon, so I grabbed my pillow and blanket and we went back to his room. We laid down on his futon and I closed my eyes.

George: So can I have a Yoshi?
Me: Sorry, buddy. Yoshi’s aren’t real.
George: Yeah, but can I have a Yoshi toy?
Me: We’ll see.
George: Is there a Super Mario Brothers Store around here?

He had taken my sleepy “We’ll see” as a “Yes” and was ready to go get it right then.

Me: I don’t think there are any Super Mario Brothers stores. Maybe in Japan.
George: Like Choo-Choo Soul?
Me: Right, like where the bullet train is.
George: OK! let’s go!
Me: It’s very far away.
George: Where is it?
Me: It’s across the Pacific Ocean.
George: Can we walk there?
Me: No, we can’t walk to Japan. It’s way too far.
George: Well, can we go there?
Me: Maybe someday.

At this point I had woken up enough to realize we had gotten way off the original topic.

Me: OK, look, we can get a Yoshi toy next time we go to Toys ‘R’ Us. Now please lay down and go back to sleep.

Two minutes later we were downstairs eating breakfast because he was “not too sleepy”. I think he wrangled a Yoshi toy and trip to Japan out of me.

by brousch at April 18, 2012 10:26 AM

April 15, 2012

Whitemice Consulting

Using GNOME Terminal Custom Commands

There are numerous terminal session managers and profile managers, etc... for use by people [typically network and system administrators] who have to SSH or telnet to lots of hosts.   But most of these aren't packages or very well maintained - fortunately there is an almost stealth solution built into the familiar gnome-terminal application.
Multiple profiles can be created in gnome-terminal and profiles can be assigned to "Run a custom command instread of my shell";  this command can be anything.  So that profile can automatically telnet or SSH to another host, potentiall even launch an application on that host (such as firing up GNU screen).

The defined profiles are conveniently available in "Files" / "Open Terminal" and "File" / "Open Tab" menu.  Simply create a profile for each host or device that you typically jump on to. If SSH is in use the automatic tie-in to the GNOME passphrase and keyring will kick in.
Generally you don't want a terminal emulator to grab your keystrokes - you want them to go to the application or session.  Under gnome-terminal's "Edit" / "Keyboard Shortcuts" you can enable F10 to activate the ring-menu which will allow you to conveniently navigate to profiles using just the keyboard.

by whitemice (noreply@blogger.com) at April 15, 2012 02:00 AM

April 14, 2012

Whitemice Consulting

Permission Denied on rpc_pipefs

On some openSUSE / SuSE hosts performing a "df" as a non-root users completes but ends with an error message of:
df: `/var/lib/nfs/rpc_pipefs': Permission denied
Text1: df error message for non-root users.

Note that this isn't a box with a mounted NFS volume but a box which is exporting an NFS volume; however, NFS is still an active subsystem.  This behavior is a bug (Bug#675385) that has since been closed. But if you can't perform updates quite yet and this message irritates you the work around is simple:
sudo chmod 711 /var/lib/nfs
Text2: Workaround
 The package related to this issue is "nfs-utils".

by whitemice (noreply@blogger.com) at April 14, 2012 02:00 AM

April 13, 2012

Whitemice Consulting

zyppering from 11.4 to 12.1

It came time to upgrade my home server from openSUSE 11.4 to openSUSE 12.1.  Everything works fine under 11.4;  the only reason for the upgrade was to stay current.  So the simplest way to perform an in-place upgrade is to zypper dup.


Step#1) Disable all your existing repositories.
zypper modifyrepo --all --disable

Step#2) Add the 12.1 repositories.

zypper ar --name "openSUSE-12.1 OSS" \
  http://download.opensuse.org/distribution/12.1/repo/oss/ \
  repo-12.1-oss
zypper ar --name "openSUSE-12.1 Non-OSS" \
  http://download.opensuse.org/distribution/12.1/repo/non-oss/ \
  repo-12.1-non-oss
zypper ar --name "openSUSE-12.1 Updates" \
  http://download.opensuse.org/update/12.1/ \
  repo-12.1-update

Step#3) Deal with third-party repositories.
Check what third-party repositories you've added and configure the 12.1 equivalent of those.  To do this just list the URI's of the currently configured repositories, remove the 11.4 versions, and re-add them replacing the 11.4 in the URI with 12.1.  gedit search-n-replace is a handy way to pull that off. 

zypper lr --uri
Text: List the configured repositories and show their URIs.

I have the Packman, libdvdcss, Mono:Community, and Database repositories in addition to the standard repos.

zypper rr "Packman Repository"
zypper rr "libdvdcss repository"
zypper rr "openSUSE BuildService - Mono:Community"
zypper rr "openSUSE BuildService - Database"
zypper ar  \
  http://download.opensuse.org/repositories/server:/database/openSUSE_12.1 \
  "openSUSE BuildService - Database"
zypper ar \
  http://download.opensuse.org/repositories/Mono:/Community/openSUSE_12.1/ \
  "openSUSE BuildService - Mono:Community"
zypper ar \
  http://opensuse-guide.org/repo/12.1 \
  "libdvdcss repository"
zypper ar \
  http://packman.inode.at/suse/openSUSE_12.1/ \
  "Packman Repository"
Text: Remove the third-party and secondary repositories.  Then add the 12.1 versions.

Step#4) Refresh the repository information

zypper refresh

Step#5) Do the in-place distribution upgrade.
I prefer to do this in the download-all-required-packages-first mode so that a network error can't break the upgrade while it is in progress.  But if you are feeling brave or have really limited disk space you can run the "dup" without any options and packages will be retrieved on-demand.
zypper dup --download "in-advance"
Have a beverage
Update 2012-04-13: A zypper guru has informed me that the --download "in-advance" is the default behavior in recent versions; possibly even in 11.4.

Step#6) Reboot.

You are done!  Your box will come back up running openSUSE 12.1.  Enjoy.

by whitemice (noreply@blogger.com) at April 13, 2012 07:39 AM

A JSONDateTime TypeDecorator

JSON doesn't provide a date or date-time construct;  therefore every application is burdened with implementing a solution of it's own for receiving date-time information over the wire.  On common issue receiving JSON and serializing that data into some type of database - but the database knows the value is a date-time and you want to be able to perform date-time like operations on the value (locally).
Fortunately if your database is accessed via an SQLAlchemy ORM you can out-source this deserialization of the the values into your ORM by creating a TypeDecorator that wraps the datetime value and knows how to recognize and parse date-time values.
This example will allow an actual date-time value to be assigned to the attribute, or a 19 or 16 character string, or a NULL (None).  Conversion happens automatically when the value is assigned, but when the value is accessed an actual datetime is always seen.
from datetime import datetime
from sqlalchemy import TypeDecorator
from sqlalchemy.types import DateTime

class JSONDateTime(TypeDecorator):
""" Allow storing a string into a datetime value, this allows
for automatically conversion of the JSON date strings into
date values"""
impl = DateTime

def __init__(self, *arg, **kw):
TypeDecorator.__init__(self, *arg, **kw)

def process_bind_param(self, value, dialect):
if value:
if isinstance(value, basestring):
if (len(value) == 19):
return datetime.strptime(value, '%Y-%m-%dT%H:%M:%S')
elif (len(value) == 16):
return datetime.strptime(value, '%Y-%m-%dT%H:%M')
elif (len(value) == 0):
return None
elif isinstance(value, datetime):
return value
raise Exception('Cannot store value "{0}" as
DateTime'.format(value))
return None

# Create a class/table

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class LogEntry(Base):
__tablename__ = 'logentry'
...
actiondate = Column(JSONDateTime)
...
Text1: A TypeDecorator class for automatically converting string datetime representations to their datetime values; and a snippet of a table utilizing the new type.

One possible improvement is to make sure to assign a time-zone to the date value as comparing zoned and nieve datetime values will raise an exception.  But to do that you need to know the timezone you expect the JSON values to represent (in most cases this isn't present in the date-time string representation). The Python datetime's replace(tzinfo={tzinfoclass}) method is used to assign time zone information to a value. [Note: time zone information in Python.... not especially graceful or fun].

by whitemice (noreply@blogger.com) at April 13, 2012 07:27 AM

April 11, 2012

Whitemice Consulting

Using Local Update Publisher; WSUS, but for you.

One of my colleagues has published an excellent and comprehensive article about using Local Update Publisher,a project hosted at SourceForge, that enables using your local Windows Update Services to distribute other and custom software. For example, your workstations can get deployments and updates of FireFox, Java, Flash, LibreOffice, etc... through the same mechanism used to stay up-to-date with Microsoft products.  LUP is a powerful tool for steamlining the managemento third-party software on you Microsoft Windows workstations;  if you've got lots of workstations be sure to check it out.

by whitemice (noreply@blogger.com) at April 11, 2012 09:00 PM

March 16, 2012

Whitemice Consulting

Discovering DLL Version With pefile

A Microsoft KB article claimed that if a specific DLL was at least a certain version that a bug reported by one of my users would be resolved.  But the user was using their computer and I dislike interrupting people's work (I know how annoying it is when someone interrupts me).  No problem; I can just grab the named DLL off their machine over the network and copy it to my home directory.   But I'm not running Windows and all file tells me is that the DLL is a 32-bit PE file.  Enter pefile, a Python module that can parse PE headers on any platform.
$ python
Python 2.7 (r27:82500, Aug 07 2010, 16:54:59) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pefile
>>> pe =  pefile.PE('lhmstscx.dll')
>>> pe.FileInfo[0].StringTable[0].entries['ProductVersion']
u'6.0.6001.18589'
Text 1: Getting the version informtion from lhmstscx.dll
Handy.  pefile can tell me lots of other things to, but in this case all I needed to know is that this DLL wasn't at the required version (despite having approved all WSUS updates for that workstation's group... but that's another story).

by whitemice (noreply@blogger.com) at March 16, 2012 09:35 AM

March 05, 2012

Whitemice Consulting

Sound Converter

A common issue is to have an audio file in one format at to need it in another for compatibility with some specific application or device.  And how to covert the file and know the quality of the result, etc...?  Well... there is a simple application called ... wait for it .... "soundconverter".  It is packaged for just about every distribution and available on openSUSE through the normal repositories.  How obvious can it get?  Apparently not so obvious I couldn't have overlooked it for a long long time.
The soundconverter application.
With soundconverter you can convert between FLAC, MP3, OGG, and WAV.  Add the files you want to covert, use preferences to select the output format, and away you go.  Nice job.

by whitemice (noreply@blogger.com) at March 05, 2012 08:11 AM

March 02, 2012

Ben Rousch's Cluster of Bleep

Remote User Group Meeting Participation

Remote participation in a meeting is one of those would-be-nice things that comes up with every user group. Most recently someone requested it via the GR Mobile Dev discussion list, which is what prompted me to finish this neglected blog post.

I’m not completely opposed to remote participation, but I have not personally seen it work at any user group meeting. Here are a few of the reason I’ve identified for why this is so:

A Bustling GRWebDev Meeting

A Bustling GRWebDev Meeting

1. Quality: Streaming a meeting via something like Ustream or Skype produces low quality video unless you have good video equipment and a lot of spare upload bandwidth. You also need a decent microphone to pick up the presenter while ignoring ambient sound. The hardware is not cheap, and when you have a dozen or more heavy Internet users in the same room, the bandwidth is precious.

2. Personnel: It can be hard to find people willing to spare the time to attend a user group meeting. Finding someone willing to spare the time so that other people can “attend” the meeting is even harder. Due to unavoidable distractions, if you are the cameraman or camerawoman, you will not get as much value out of the presentations as a fully-engaged audience member.

3. For actual two-way participation you need some kind of physical presence at the meeting. A laptop sitting on a table in the back of the room isn’t going to be loud enough to be noticed, which means someone at the meeting has to babysit the remote participants to relay their interactions. Similar to reason #2, you’re going to have trouble finding that babysitter.

Obviously these problems are not insurmountable, but so far they have been enough to prevent any of the meetings I attend from doing remote participation. That doesn’t mean it can’t work, it just means I haven’t seen it. If you’re willing to commit to being the streaming cameraman, microphone wrangler, remote presence babysitter, or whatever else needs to be done to get it to work, please get a hold of me.

Now if you’ll indulge me a bit, I will explain why I think remote participation isn’t really even valuable for a user group.

To be frank, much of the information coming out of the typical user group meeting is not groundbreaking. In most cases, you can find presentations with similar information on YouTube, Slideshare, or SpeakerDeck.

You don't get this by remote participation.

You don't get this by remote participation.

As an attendee, what really makes a user group meeting valuable is the opportunity to meet and engage with other enthusiastic local developers – something you will not get via remote participation. If you attend the meetings and put a little effort into engaging and talking to people there, you’ll make new friends and you’ll build a valuable network of peers. These are people you can consult when you need a new job, when you’re learning a new technology, or when you need to refer someone to someone you trust.

I was a hermit for a decade before I put aside my fears and started coming to user group meetings. It’s worth it. Trust me. You’ll get to know some awesome people.

As a viewer, I personally find a streamed presentation less valuable than one which has been recorded and posted to YouTube. If I had that particular time slot free, why would I watch from home instead of going to the event in person? Why passively consume the meeting instead of actually shaking hands with the people presenting? Also, if it’s on YouTube, I can watch at my own pace on my own time while I’m doing something else, such as riding the exercise bike. The recorded and uploaded version just makes more sense than a streamed version.

To this end, I have started recording some of the meetings I attend and putting those videos up on YouTube. For each two hour meeting I end up with about 1.5 hours of presentation footage. Afterwards, it takes me about two hours of editing, then one night to render and another to upload (1.5 hours of HD video is big). I have decided to take on this task because I think the videos will be good for helping groups to promote themselves. Hopefully some folks who have heard of the groups but don’t know if it’s worth taking the time to attend can find out what they’re missing. I also think that having the meetings online may bring in bigger presenters because we can give them a little bigger audience.

Now I know a lot of you think you don’t have the time to attend a meeting. Hey, I have a wife and a kid, so I know it’s hard to justify being away from home. Do it anyways. Being gone a few nights a month does not make you a bad parent, husband, or wife. Swap a night with your spouse and you each get some one-on-one time with the kids, as well as a night off from them. I’ll bet your parents want to spend time with their grandkids. By the time the meeting is over, the kids are in bed, so you might as well go have a drink at the after-party – that’s where you really get to meet the interesting people.

To sum up, I don’t like remote participation for several reasons: it distracts organizers, volunteers, and attendees; a recorded copy of the presentation is better than a streamed version; and you will get none of the real value of a user group meeting by sitting at home.

by brousch at March 02, 2012 09:54 PM

February 29, 2012

Ben Rousch's Cluster of Bleep

My Principle

I watched Bret Victor’s talk “Inventing on Principle” last night and since then I have been thinking about what My Principle could be.

Bret Victor – Inventing on Principle from CUSEC on Vimeo.

The most troubling “bug” I see today is computing devices and platforms which cannot be used to create programs that run on themselves. Today’s tablets and smartphones have more processing power, more pixels, and more RAM than a desktop computer from a few years ago. Yet you need a laptop or desktop to develop for Android, you need a Macintosh to develop for iOS, and you need a Windows PC to develop for whatever Microsoft’s mobile operating system is called these days.

This means our children are growing up with computing devices they can’t easily program. An entire layer of complexity, annoyance, and additional cost has been added to the programming environment. I can’t just hack away at a program on my Android tablet without bringing a whole other computer into play. Schools are giving iPads to every kid, but they can’t actually learn to program with them. They can’t discover how the programs they are using are created or how they work.

This does not make sense to me.

The web is in a similar situation. You can’t make a website from within a web interface. Why can’t we bootstrap a website from a web small stub, and then develop the site from within that web environment?

I know of one in-road which has been made on Android: the SL4A project lets you use various scripting languages to program right on the device, but it’s cumbersome when you go beyond basic features.

I think WordPress comes pretty close to this on the web side, which may explain it’s popularity, especially among non-hardcore programmers.

I’m going to put a little more thought into this situation and see what emerges. If you know of any existing solutions to this problem that I’ve overlooked, please let me know.

And if you need want motivation for determining and following Your Principle, watch the video.

by brousch at February 29, 2012 08:16 PM

February 27, 2012

Whitemice Consulting

Don't forget about /etc/shells

So I'm attempting, over and over, to authenticate to a new service I've configured on a host.  And it keeps on telling me "Login incorrect".  Grr... Argh...

Then I look in the service's PAM configuration [/etc/pam.d/{serviceName}].  And there I see that in addition to the typical include of the host's default password-auth stack it has added a requirement for pam_shells.  Ah ha!  I remember something about that from approximately a hundred or so years ago.

pam_shells restricts authentication to accounts whose UNIX shell attribute is one of the shells listed in the text file /etc/shells.
Aside:  The UNIX shell of an account is the seventh, and last, column in the traditional /etc/passwd file or the loginShell attribute of the account attribute if you are using the RFC2307 LDAP schema.

Since my account was set to use a weirdo shell ["/bin/ksh"] that had not been added to /etc/shells my authentication was failing to satisfy this PAM stack.  So -
echo "/bin/ksh" >> /etc/shells
- and away I go.

by whitemice (noreply@blogger.com) at February 27, 2012 08:51 AM

February 22, 2012

OpenGroupware (Legacy and Coils)

Upcoming Reaper Enhancements

The next version of OpenGroupware Coils [0.1.47] provides a new much enhanced workflow reaper. The reaper is the component responsible for cleaning up old and completed workflow processes. Previously you have been able to keep processes that successfully completed around by setting an object property on the related route - you can still do that. But there is also new functionality for automatically expiring processes after a specified number of days as well as moving processes into an archived state. All the properties used to control reaping are still created on the route and then applied to the processes derived from that route.
The server itself has one default that influences reaper behavour: OIEDefaultProcessExpirationDays. This specifies the number of days that failed or completed but preserved processes will be kept before they are reaped. The default value if not configured is 15 days.

Retention For A Failed Process
When a process fails it is kept until either the value of OIEDefaultProcessExpirationDays or the value of the {http://www.opengroupware.us/oie}expireDays object property days past the processes' failure date.  The object property must exist on the route entity from which the process was derived.

Retention For A Completed Process
When a process completes successfully it is will be reaped shortly thereafter unless the {http://www.opengroupware.us/oie}preserveAfterCompletion object property on the related route is set to "YES". If the process is preserved it is kept for the same number of days as if it had failed [the number of days past the processes' completed date plus either the server's OIEDefaultProcessExpirationDays default or the related route's {http://www.opengroupware.us/oie}expireDays object property.

Aside: If the reaper encounters an {http://www.opengroupware.us/oie}expireDays object property which is not an integer, most likely due to administrative error, it will interpret such a value as three times the server's OIEDefaultProcessExpirationDays value.

To Delete Or To Archive
Once a process is reaped, either because it is completed and not preserved or preserved but has reached the expiration age, the reaper looks for an object property of {http://www.opengroupware.us/oie}archiveAfterExpiration on the related route. If this object property is found and has a value of "YES" the reaper will archive the route rather than deleting it.


Archived Processes
An archived process does not appear in the normal process lists retrieved via REST from the /dav/Workflow/.ps URL. In the WebDAV presentation they are moved to the Archived folder that now appears in a Route's WebDAV folder. Archiving processes allows a large number of processes to be kept in the system without burdening clients and user's with every larger and longer processes lists.

At this time archived processes remain archived forever - or until the administrator deletes them. Processes in an archived state can still be retrieved via their object id and deleted based on normal permissions.

With the additional flexibility of the OpenGroupware Integration Engine reaper component available in 0.1.47 it should be possible to meet most needs to retain workflow data for assurance, archival, and analytic purposes.


Example #1
adam> list-properties --objectid=16358290
objectProperty: {http://www.opengroupware.us/oie}expireDays = 7
objectProperty: {http://www.opengroupware.us/oie}archiveAfterExpiration = NO
objectProperty: {http://www.opengroupware.us/oie}preserveAfterCompletion = NO
objectProperty: {http://www.opengroupware.us/oie}routeGroup = hederaJournalizedETL
objectProperty: {http://www.opengroupware.us/oie}singleton = YES
Text 1: The properties for route objectId#16358290 as seen using snurtle's list-properties command.  These properties indicate that a process derived from this route will be kept for seven days if it fails.  If it completes successfully it will not be preserved but eligible for immediate reaping.  Once reaped the process will be deleted, not archived; all information related to the process will be expunged from the workflow engine.
Example #2

adam> list-properties --objectid=15177350
objectProperty: {http://www.opengroupware.us/oie}archiveAfterExpiration = YES
objectProperty: {http://www.opengroupware.us/oie}expireDays = 2
objectProperty: {http://www.opengroupware.us/oie}preserveAfterCompletion = YES
Text 2: The properties for route objectId#15177350 as seen using snurtle's list-properties command.  These properties indicate that a process derived from this route will be kept for two days regardless of success or failure  Once reaped the process will be moved to an archived state. In that state it will be hidden from normal process listings but all data related to the process will be retained in the workflow engine.

by whitemice (noreply@blogger.com) at February 22, 2012 05:00 AM

February 20, 2012

Whitemice Consulting

Sequestering E-Mail

When testing applications one of the concerns is always that their actions don't effect the real-world. One aspect of that this is sending e-mail; the last thing you want is the application you are testing to send a paid-in-full customer a flurry of e-mails that he owes you a zillion dollars. A simple, and reliable,  method to avoid this is to adjust the Postfix server on the host used for testing to bury all mail in a shared folder.  This way:
  • You don't need to make any changes to the application between production and testing.
  • You can see the message content exactly as it would ordinarily have been delivered.
To accomplish this you can use Postfix's generic address rewriting feature;  generic address rewriting processes addresses of messages sent [vs. received as is the more typical case for address rewriting] by the service.  For this example we'll rewrite every address to shared+myfolder@example.com using a regular expression.

Step#1

Create the regular expression map.  Maps are how Postfix handles all rewriting; a match for the input address is looked for in the left hand [key] column and rewritten in the form specified by the right hand [value] column.
echo "/(.)/           shared+myfolder@example.com" > /etc/postfix/generic.regexp
Text 1: Create the new regular expression map.
Step#2

Configure Postfix to use the new map for generic address rewriting.
postconf -e smtp_generic_maps=regexp:/etc/postfix/generic.regexp
Text 2: Enable generic rewriting using the regular expression map.

Step#3

Tell Postfix to reload its configuration.
postfix reload
Text 3: Signal the running Postfix service to reload its configuration.

Now any mail, to any address, sent via the hosts' Postfix service, will be driven not to the original address but to the shared "myfolder" folder.

by whitemice (noreply@blogger.com) at February 20, 2012 06:14 AM

February 15, 2012

Whitemice Consulting

Complex Queries With SQLAlchemy (Example#1)

There are lots of examples of how to use SQLAlchemy to provide your Python application with a first-rate ORM. But most of these examples tend to model very trivial queries;  but the real power of SQLAlchemy, unlike many ORM solutions, is that it doesn't hide / bury the power of the RDBMS - and if you aren't going to use that power why bother with an RDBMS at all [Aren't NoSQL solutions the IT fad of the year? You could be so hip!].  So in this post I'll provide a useful non-trivial query and how to perform the a complex query - only better - using SQLalchemy.
So first, a query:
SELECT process.process_id, op1.value_int, op2.value_string, op3.value_string
FROM process
  INNER JOIN route ON ( route.route_id = process.route_id )
  LEFT OUTER JOIN obj_property op1
    ON ( op1.obj_id = route.route_id AND
         op1.namespace_prefix = 'http://www.opengroupware.us/oie' AND
         op1.value_key = 'expireDays' )
  LEFT OUTER JOIN obj_property op2
    ON ( op2.obj_id = route.route_id AND
         op2.namespace_prefix = 'http://www.opengroupware.us/oie' AND
         op2.value_key = 'preserveAfterCompletion' )
  LEFT OUTER JOIN obj_property op3
    ON ( op3.obj_id = route.route_id AND
         op3.namespace_prefix = 'http://www.opengroupware.us/oie' AND
         op3.value_key = 'archiveAfterExpiration' )
WHERE process.db_status != 'archived'
  AND process.state IN ( 'C', 'F', 'Z' )
  AND process.status != 'archived';
Text 1: A sophisticated query that exploits the power of the PostgreSQL database engine.

This query returns the process_id value from the process table and then some values from multiple records from the table obj_property - if those records exist - correlated via an intermediate table route.  Complex, but also fast!  
Aside:With the correct DDL when creating the database PostgreSQL also enforces the integrity of the relations, maintains usage statistics, and provides multiple levels of atomicity.  I certainly wouldn't want to have to do all that myself.
So how to model such a query in SQLAlchemy?  First realize that the result is actually going to be better than what straight SQL / DBAPI would give us.  Instead of some values the query will return real objects, the advantage of this is the amount of code eliminated by having to do things based on ids or primary keys.  This example assumes that the ORM entities Process, Route, and ObjectProperty have already been described - there is lots of documentation about how to declare the relations between your tables and your objects using the declarative style.
from sqlalchemy.orm   import  aliased
from sqlalchemy       import and_, or_

db = ctx.db_session()

op1 = aliased(ObjectProperty)
op2 = aliased(ObjectProperty)
op3 = aliased(ObjectProperty)

q = db.query( Process, op1, op2, op3 ).\
       join( Route, Route.object_id == Process.route_id ).\
       outerjoin( op1, and_( op1.parent_id == Route.object_id,
                             op1.namespace=='http://www.opengroupware.us/oie',
                             op1.name=='expireDays' ), ).\
       outerjoin( op2, and_( op2.parent_id == Route.object_id,
                             op2.namespace=='http://www.opengroupware.us/oie',
                             op2.name=='preserveAfterCompletion' ), ).\
       outerjoin( op3, and_( op3.parent_id == Route.object_id,
                             op3.namespace=='http://www.opengroupware.us/oie',
                             op3.name=='archiveAfterExpiration' ), ).\
       filter( and_( Process.state.in_( [ 'C', 'F', 'Z' ] ),
                     Process.status != 'archived' ) )
Text 2: The same query as above, only expressed via the SQLAlchemy ORM.  But instead of returning values it returns live objects.
The SQLAlchemy aliased method declares multiple references to ObjectProperty that can be used independently: op1, op2, and op3.  The other advanced technique is to use the outerjoin method to relate the need for a LEFT OUTER join.
The results of this query will be tuples of four elements; the first being a Process object and the second, third, and fourth will either be ObjectProperty objects if the concomitant outer join identified a record or None if no record matched the join. The lovely upside of this is that the query results can be processed using a straight forward for-each construct:
for process, expire_days, preserve_after, archive_after in q.all():
   if expire_days:
       ....
Text 3: Iterate over the query results; the first step depends if the op1 is an object (a record matched the first outer join).
Personally I find the ORM code to be easier to visually parse than the native SQL. Especially if you need to build the query dynamically or modify it based on the applications needs - since q is an object additional filter and join conditions can continue to be added.  Imagine trying to do that with straight SQL?
q = q.filter(Process.owner_id == 10100)
q = q.limit(150)
Text 4: Add one more filter expression to the queries WHERE clause and limit the query to 150 results.
Another advantage to this method is that SQLAlchemy can adapt it's dialect to the specific back-end if, for example, you are stuck using a database other that PostgreSQL.  Without such an adaptive layer using anything other than the most trivial queries becomes daunting do to slight but important differences in how various engines express joins and nested queries.

by whitemice (noreply@blogger.com) at February 15, 2012 07:40 PM

February 09, 2012

Whitemice Consulting

Configuring Postfix As An SMTP Client

Every host needs to send mail; not just users.  Hosts send mail for a variety of reasons - from cron jobs, log watchers, error and exception reports, lots and lots of reasons.  But mail sent by hosts should be as secure as mail sent by users at least to the degree you trust the facilities generating the mail.  To achieve that the host's Postfix instance should encrypt it's traffic to the central SMTP relay and it should authenticate itself - just like a user.  Fortunately doing so is pretty straight-forward.  For this example I'm assuming the central SMTP server is smtp.example.com and you have a username/password you need to authenticate. 

A note about SASL mechs

Whether authentication is performed using DIGEST, PLAIN, CRAM, etc... doesn't really matter.  The only caveat for the type of authentication is that you need to have the appropriate SASL library installed; so for plain you need to "yum install cyrus-sasl-plain".  If you get to the end and don't have an appropriate SASL library installed for a type of authentication the central SMTP server supports you'll see messages like "SASL authentication failure: No worthy mechs found".  It is the central SMTP server that determines what authentication methods are acceptable - your client has to be able to match at least on of the methods it supports.

Step #1 : Point to the central SMTP server

Configure the Postfix instance to only listen to the local interface and to send all mail, regardless of destination, to the central relay.
postconf -e inet_interfaces=localhost
postconf -e relayhost='[smtp.example.com]'
Text 1: Setting the central SMTP server (relayhost)
Step #2 : Enable authentication & encryption

Of course you'll want to encrypt the traffic and the relay host will probably only permit authentication over an encrypted connection anyway.
postconf -e smtp_sasl_auth_enable=yes
postconf -e smtp_use_tls=yes
postconf -e smtp_tls_note_starttls_offer = yes
Text 2: Enable TLS & authentication
If the site is using their own CA certificate to create SSL certificates then that CA certification must be available on the host in order to verify the host certificate of the SMTP relay.
postconf -e smtp_tls_CAfile=/path/to/the/cacert.pem
Text 3: Set the path to the CA certificate
Step #3 : Establish the authentication credentials

Now the SMTP server needs some credentials.  These are written to a file and then a Postfix map is generated from that file. The format of the file is the host name of the remote, whitespace, and then the username and password delimited by a colon.  Note that the hostname must match the actual hostname of the remote or the local Postfix instance won't attempt to login - it will think it doesn't have credentials. The permissions on the sasl_passwd and sasl_passwd.db files should be secured so that only user root & group mail have access.

echo "smtp.example.com username:password" > sasl_passwd
postmap hash:/etc/postfix/sasl_passwd
postconf -e smtp_sasl_password_maps=hash:/etc/postfix/sasl_passwd
chown root:mail sasl_passwd sasl_passwd.db
chmod 740 sasl_passwd sasl_passwd.db
Text 4: Creating the sasl_passwd map
Optional Extra Paranoia

My personal preference, for a bit of added paranoia, is to also set the immutable flag of the three security sensitive files.
chattr +i cacert.pem sasl_passwd sasl_passwd.db
Text 5: Making the sensitive file immutable.

A file set as imutable canntot be modfied, deleted, renamed, or linked to.  Not even by root - at least not until the immutability flag is explicitly removed [chatter -i files].  This protects the file from being modified or deleted unintentionally as well as making them that much more difficult to modify maliciously.

Step #4: Test
Now you should be able to send some mail;  this is most easily accomplished with the mail command [which is provided in the mailx package].  Watch the /var/log/maillog file to see your message go; or see any errors. If you see messages like "certificate verification failed for ..." then Postfix doesn't accept the validity of the central SMTP relay's certificate.  Either the CA cert specified in Step#2 is invalid or the permissions are incorrect and Postfix can access the file.
When you receive the mail you've sent in your INBOX you can look at the headers and you should see something very much like:
Received: from client.example.com (client.example.com [192.168.1.70]) \
  (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No \
  client certificate requested) (Authenticated sender: smtpclient) by \
  smtp.example.com (Postfix) with ESMTP id 5FD712000C for \
  ; Wed,  8 Feb 2012 12:05:19 -0500 (EST)
Text 6: Example header from a secure and authenticated e-mail.
You server is now securely sending messages.

by whitemice (noreply@blogger.com) at February 09, 2012 05:50 AM

February 08, 2012

Whitemice Consulting

Installing PDO_INFORMIX on CentOS6

Step#1 : Install the Informix SDK / Client

This is as simple as copying the files to /opt/informix or using one of the various install methods provided.  But beyond that it is necessarily to initialize the required environment variables.  The simplest way to set the environment variables is to create an informix.sh profile script in /etc/profile.d - these scripts are executed by /etc/profile whenever a session is created [such as when a user logs in].  Additionally you'll need to set these same variables in /etc/sysconfig/httpd so that they are set in Apache's environment when started by the system start-up scripts.
$ (cat << EOF
export INFORMIXDIR=/opt/informix
export DBDATE='Y4MD-'
export INFORMIXSERVER=YOURINSTANCENAME
EOF
) > /etc/profile.d/informix.sh
Text 1: Creating /etc/profile.d/informix.sh
YOURINSTANCENAME needs to be defined in /opt/informix/etc/sqlhosts.  Your method of installing the SDK may or may not have set that up for you.

The system library path must also be extended to include the directories containing the SDK's libraries.
$ ( cat << EOF
 /opt/informix/lib
 /opt/informix/lib/cli
 /opt/informix/lib/esql
 /opt/informix/lib/client
 /opt/informix/lib/csm
 /opt/informix/lib/dmi
 EOF
 ) > /etc/ld.so.conf.d/informix.conf
Text 2: Extending the system's library path
 If the library path is not configured correctly applications, included httpd, will not be able to load the Informix libraries.  At this point the library cache can be refreshed by executing the /sbin/ldconfig command.  Once that has been performed either log out and back into the server, or just reboot the server, to verify that upon logging in you have the INFORMIXDIR, INFORMIXSERVER, and DBDATE variables in your enviroment.

Step#2 : Build the Informix PDO driver.

In order to build PHP PECL modules you must have php-devel, make, and gcc installed on the server.
$ pecl download PDO_INFORMIX-1.2.6
$ tar xzf PDO_INFORMIX-1.2.6.tgz
$ cd PDO_INFORMIX-1.2.6
$ phpize
$ configure
$ make
Text 3: Building PDO_INFORMIX
If your Informix SDK is installed correctly and you've properly initialized the environment everything should be found automatically and build without complaint.  Now move the PDO driver into place and inform the PHP interpreter that it needs to load the library.  Here we perform a dirty trick of first loading the base pdo.so library.  This shouldn't be necessary and PHP will grumble about it upon initialization, but it works around some wackiness regarding PDO versions.  Without this line pdo_informix.so will refuse to load because PDO isn't loaded yet because the need for PDO isn't automatically discovered.
$ cp /tmp/PDO_INFORMIX-1.2.6/modules/pdo_informix.so /usr/lib64/php/modules/
$ ( cat << EOF
extension=pdo.so
extension=pdo_informix.so
EOF
 ) > > /etc/php.d/informix.ini
Text 4:  Install and register PDO_INFORMIX
Now we can try to start/restart the Apache service and see if our PDO module is available: service httpd restart.  But it won't work. The loading of the Informix SDK by Apache will be blocked by SELinux's security policy.

Step#3 : Provision SELinux


PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/pdo_informix.so' - libifcli.so: failed to map segment from shared object: Permission denied in Unknown on line 0
The message in /var/log/httpd/error_log indicating that loading the library failed with a "permission denied"; regardless of what you set the permissions too.
Text 4: SELinux blocking the loading of libifcli.so
The solution is not to disable SELinux; SELinux is your over-protective big brother.  Maybe annoying to have around sometimes, but worth it for those time when you need to take a short cut through a dark musty alley.  The correct solution is just to label the required library as a known and approved shared library.
$ chcon -t lib_t /opt/informix/lib/cli/libifcli.so
Text 5: Applying the appropriate label to libifcli.so
Step#3 : Get coffee

Restarting Apache know and you should see the pdo_informix driver available in phpinfo() output.  Also double check that INFORMIXDIR, INFORMIXSERVER, and DBDATE appear in the "Enviroment" section of phpinfo; without these variables the PDO driver will not be able to find your informix instance.

From here on out it is pretty much the web developer's problem.

by whitemice (noreply@blogger.com) at February 08, 2012 10:33 AM

February 07, 2012

Whitemice Consulting

Renaming The Volume Group Containing /

Almost every server I work with is a virtual machine;  accordingly I like to do one small install with all the packages that I always want [like pam-nss-ldapd, snmp-utils, dstat, etc...] but don't install by default.  Then I make sure VMware tools is installed an operational.  From that point forward I can just clone that one VM and add to it when I want a new instance of something.
The only downside to this is that all the machines end up with the same volume group name: typically VolGroup0 or some such thing.  That's ugly and limits the ability to move volume groups around - these are virtual disks after all.  Renaming a volume group is straight forward:
vgrename /dev/VolGroup0  /dev/Carazon
Text 1: Rename the volume group VolGroup0 to be named Carazon.
 Hold it!  Now the machine won't boot anymore!  You get the pleasure of rebooting into "Kernel Panic:  blah blah blah".  If you rename the volume group containing the root filesystem and swap space you also have to update that name in two places -
  1. In /etc/fstab.  This one is obvious and I usually remember.
  2. In /etc/grub.conf.  Otherwise the kernel tries to mount the root file-system using the old volume group name.
Done this so many times... always forget.

by whitemice (noreply@blogger.com) at February 07, 2012 04:36 PM

OpenGroupware (Legacy and Coils)

An Introduction to OIE Tables

The OIE Table entity provides a simple means to embed look-ups, filters, and translations into workflows. The principle of a Table is that it always receives a value and returns a value - a look-up.

Table definitions are presented via WebDAV in the /dav/Workflow/Tables folder as simple YAML files; they can be created and edited using your favorite text editor.  If you are familiar with OIE Format definitions the Table definition should seem very familiar. Tables are identified by their unique name which is specified by the name attribute of their YAML description.

StaticLookupTable
The static look-up table provides a method to do simple recoding of data without relying on external data-sources such as an LDAP DSA or SQL RDBMS. The definition of a StaticLookupTable provides a values dictionary where input values are looked up and the corresponding value returned. The optional defaultValue directive may specify a value to be returned if the input value is not found in the values table; if no defaultValue is specified the table will return a None.
class: StaticLookupTable
defaultValue: 9
values: { 'ME1932': 4,
          'Kalamazoo': 'abc' }
name: TestStaticLookupTable
Text 1: A StaticLookupTable that returns 4 for the input value "ME1932", and "abc" for the input value "Kalamazoo".  Any other input value results in the value 9.
PresenceLookupTable
A presence look-up table contains a list of static values. It returns a specified value if the input value matches one of the values stored in the table; otherwise it returns an alternative value. Presence look-up tables are most commonly used when a small and known set of values needs to used to filter a set of data.
class: PresenceLookupTable
name: BankeCodeExclusionTable
returnValueForFalse: true
returnValueForTrue: false
values: [ME1932, Kalamazoo, 123]
Text 2: A PresenceLookupTable that returns boolean false for the input values "ME1932", "Kalamazo", and 123; returns boolean true for all other input values.
SQLLookupTable
An SQLLookupTable permits the translation or look-up of values using an SQL data source defined in the OIESQLSources server default. The table definition must at the minimum define SQLQueryText and SQLDataSourceName directives. Within the SQLQueryText value the "?" is substituted for the input value; the first column of the query result is the return value of the table. If the query identifies no rows then a None value is returned from the table.
SQLDataSourceName: mydbconnection
SQLQueryText: 'SELECT CASE WHEN COUNT(*) = 0 THEN
    ''True'' ELSE ''False'' END  FROM bank_code_exclusion WHERE bank_code
    = ? AND ex_service_followup = ''Y'';'
class: SQLLookupTable
doInputStrip: true
doInputUpper: true
doOutputStrip: false
doOutputUpper: false
name: ServiceFollowUpExclusionTable,
useSessionCache: true
Text 3: An example SQLLookup table which uses the data-source "mydbconnection" as defined in the OIESQLDataSources server default.
The optional directives: doInputStrip, doInputUpper, doOutputStrip and doOuputUpper, which all default to false, allow the input and the output values to be changed to upper case and stripped of white-space.  Converting a value to upper case may be useful in the case where a database backend itself does not support case-insensitive compare.  Trimming whitespace on input values can protect from attempting to look-up padded strings and output trimming is useful for database engines that always return strings values defined like CHAR(30) as padded values.


Using Tables
In Python code using a table is as simple as loading the class and calling the lookup_value method.  However the Table performs the look-up is entirely encapsulated in appropriate Table class [SQLLookupTable, StaticLookupTable, ...]
table = Table.Load(name)
return table.lookup_value(value)
Text 4: How to use a table to look-up values in Python code.
More commonly Table lookups are going to be performed within workflow actions such as maps and transforms. When performing an XSLT transform using any table is available via the tablelookup OIE extension method; this allows values from the input stream to be easily used as lookup-values facilitating translation of ERP and other codes/abbreviations between disparate applications.
<xsl:template match="row">
</xsl:template>
<xsl:if test="total_charges>1000">
</xsl:if>
<xsl:variable name="include" select="oie:tablelookup('ServiceFollowUpExclusionTable',string(bank_code))"/>
<xsl:if test="$include='True'">
<row>

Text 5: This snippet of an XSLT transform demonstrates how to use a Table lookup from with an stylesheet.
Overall Tables provide a simple and elegant way to automate all the codes that need to be inserted and translated in the wide variety of documents processed by the workflow engine as well as providing a means to easily implement dynamic filtering.

by whitemice (noreply@blogger.com) at February 07, 2012 01:06 PM

Whitemice Consulting

Identifying The Hottest Tables (Informix)

Yesterday I posted about how to identify the hottest table in a PostgreSQL database.  Pretty much the same functionality is available for administrators of Informix databases as well; this kind of information is found in the "sysmaster" database which is the database engine's own database.
SELECT
        TRIM(dbsname) || ':' || TRIM(tabname) AS relation_name,
        isreads AS records_read,
        pagreads AS page_reads,
        iswrites AS records_inserted
        bufwrites AS buffered_writes
FROM sysmaster:sysptprof
ORDER BY isreads DESC;
Text 1: List the basic read and write statistics for objects in the engine's databases.
This will list a record for every object in the database including indexes; the ratio of ISAM operations vs. buffer page operations can give you a hint as to the effectiveness of your server's configuration. If the ratio is very low for busy object your buffer pool is possibly too small.
If you are interested in the counts of various query operations the sysptprof table also provides the following values:
  • isrwrite - The number of records updated.
  • isrdelete - The number of records deleted.
These counters will reset whenever the database server is restarted. In versions 11.7 and later of the Informix persistent values are available from the sysmaster database.
Many more sysmaster queries can be found in the Informix Wiki.

by whitemice (noreply@blogger.com) at February 07, 2012 06:09 AM

Identifying The Hottest Tables (PostgreSQL)

In recent versions of PostgreSQL there is a magical view called pg_stat_user_tables which provides per table information on usage; there is one row per table and eight counters per row.
Fields
relname - The name of the table.
seq_scan - The number of sequential scans that have been performed on the table. A sequential scan is a read of the table from beginning to end, either because the table is very small or no indexes were available that could satisfy the filter criteria in an efficient way. Sequential scans are probably the most expensive operation the database server performs, some are however unavoidable. If proper indexing cannot resolve the need to sequentially scan a table it is imperative that the PostrgeSQL configuration provide enough resources to maintain a high cache rate.
seq_tup_read - The number of rows processed through sequential scans. This is not the number of records returned to the applications as results but the number of records processed in order to create the result set, which is probably a significant subset of this number. For example, if a query returns ten records but requires a sequential scan of the table then this value will increase by the number of records in the table, not by ten.
idx_scans - The number of indexes scans of the tables.
idx_tup_fetch - The number of rows processes through indexed scans. As with seq_tup_read this is not the count of records returned as the results of queries but those evaluated for queries due to index entries.
seq_tup_read - The number of records processed in order to create the result set of a query, not the number of records returned to the applications.
n_tup_ins - The number of rows inserted into the table.
n_tup_upd - The number of rows updated.
n_tup_del - The number of rows deleted.
Using this view an administrator can isolate the busiest tables in the database.
SELECT relname AS table_name,
seq_tup_read, idx_tup_fetch
FROM pg_stat_user_tables
WHERE (seq_tup_read + idx_tup_fetch) > 0
ORDER BY records DESC LIMIT 10
Text 1: Query to return the ten hottest tables with their sequential and index tuple fetch values.
These results will reveal both table usage and the effectiveness of your indexes.  If you have lots of sequential scans occurring then the query engine isn't finding indexes that match the queries being performed.
     table_name                           seq_tup_read      idx_tup_fetch
doc                                       1,423,407,729,074    349,028,985,971
job_history                                        71,378,301     4,213,364,118
job_history_info                               74,454,363      4,207,594,850
date_company_assignment           31,059,671      1,305,469,897
enterprise                                    3,551,311,871     1,083,015,878
date_x                                               12,884,498        982,418,723
object_acl                                  15,942,621,939        137,179,721
job                                               39,956,712,914          46,912,825
project_info                                 1,709,329,011                         23
team                                           1,141,035,688                            0
Text 2: Example results.
In these example results it is apparent that the table doc is one of the hottest objects and while many records are being identified using index entries there is also a very large number of sequential processes occurring.  This may be because either the indexes do not match the queries being performed or the cardinality of the indexed values is too low. Now we know where to look. 
So don't grope about speculating about how to improve database performance or scalability - ask where to look, PostgreSQL wants to help you.  Much more information can be found at the PostgreSQL stats monitoring documentation.

by whitemice (noreply@blogger.com) at February 07, 2012 05:55 AM

February 06, 2012

Whitemice Consulting

Integrating Postfix And CLAMAV

The clamav-miler is packaged by most distributions in their "clamav" package can be used in conjunction with Postfix to protect your network from malware embedded in SMTP traffic. Integration of CLAMAV and Postfix involves four steps:

  1. Configuration and enabling of the clamd service.
  2. Updating the CLAMAV malware database and enabling the freshclam service
  3. Configuration and enabling of the clamav-milter service. Current versions of the clamav-milter require connectivity to the clamd daemon through either a local UNIX socker or a TCP/IP socket.
  4. Configuration of Postfix to utilize the available clamav-milter service.
Step#1 : Enabling the clamd service
LocalSocket /var/lib/clamav/clamd-socket
LogFacility LOG_MAIL
LogSyslog yes
PidFile /var/lib/clamav/clamd.pid
TCPAddr 127.0.0.1
TCPSocket 3310
User vscan
Text 1: Typical settings overridden from defaults in /etc/clamd.conf

The clamd daemon typically reads its configuration from the /etc/clamd.conf file. Most importantly this file specifies, via the TCPSocket and TCPAddr directives, on what IP port and address the service listens for connections. These directives should be set to values appropriate for the host and which will be reachable by the clamav-milter. If the clamav-milter and the clamd daemon will be running on the same host the clamd service can be configured to listen to the localhost address [127.0.0.1] to avoid any potential network firewall and traffic filtering issues.
The clamd.conf file also provides many other tunable values but almost all of these should be appropriate at the distributions defaults.
Once configured the clamd service must be started and enabled for automatic start following the system's boot-up sequence; on RPM based systems this is typically achieved using the service and chkconfig commands.

Step #2 : Enabling the freshclam service

The freshclam service is an instance of the freshclam command line tool started with the “-d” option which runs the command in daemon mode. Whether started from the command-line or running in daemon mode freshclam will read its configuration from the /etc/freshclam.conf file. When running the freshclam daemon will periodically check the CLAMAV project mirrors for new malware signatures and update the local database used by the clamd scanning service. The freshclam daemon should run as the same user context as the clamd service; the typical way to ensure this is to synchronize the values of DatabaseOwner in /etc/freshclam.conf and User in /etc/clamd.conf. The frequency which freshclam will check for new patterns is controlled by the Checks directive – the default is 12 [times a day], this value should be sufficient in most cases. When database update succeeds the freshclam service will notify the clamd service that newer patterns are now available [for this to work the NotifyClamd directive must indicate the correct path to the current clamd configuration file].
DatabaseMirror database.clamav.net
DatabaseOwner vscan
HTTPProxyPort 3128
HTTPProxyServer proxy.example.com
LogFacility LOG_MAIL
LogSyslog yes
NotifyClamd /etc/clamd.conf
OnErrorExecute /usr/local/bin/malware_update_fail.sh
OnUpdateExecute /usr/local/bin/malware_update_ok.sh
PidFile /var/lib/clamav/freshclam.pid
UpdateLogFile /var/log/freshclam.log
Text 2: Example /etc/freshclam.conf file (comments removed)

The most important considerations in configuration of freshclam is if your network configuration requires use of an HTTP proxy server in order to access the CLAMAV mirrors for updates and if you need to configure some form of notification concerning success or failure of the pattern updates – a security focused service like a malware milter doesn't help anyone if it is silently failing in the background.
The HTTPProxyPort and HTTPProxyServer directives allow an HTTP proxy to be specified; freshclam will use this proxy for all mirror requests whether running as a command-line utility or in daemon mode. Should your proxy require a username/password for authentication these can be provided using the additional HTTPProxyUsername and HTTPProxyPassword directives. However it is simpler and more reliable to simply approve the “database.clamav.net” domain and sub-domains on your HTTP proxy service; all mirror requests will be made to those domains.
For notification of successful or failed updates the OnUpdateExecute and OnErrorExecute directives are used respectively. Whatever command is specified here will execute in the security context of the DatabaseOwner. A useful approach is to enable the log file via the UpdateLogFile directive and have the tail-end of that file mailed to a responsible party such as a help-desk or system-administrator for periodic verification that the service is operational.
#!/bin/sh

tail -25  /var/log/freshclam.log \
 | mail -s "[NOTICE] Malware Database Update Successful" \
    -r milter@example.com helpdesk@example.com
Text 3: A simple example script that might be used for OnUpdateExecute
The proper operation of freshclam can be tested by simply executing the freshclam utility on the command-line; it should check the mirrors and download any new patterns without an error message. Once configured and tested the freshclam service must be started and enabled for automatic start following the system's boot-up sequence.

Step #3 : Enabling the clamav-milter service
ClamdSocket tcp:127.0.0.1
LogFacility LOG_MAIL
LogSyslog yes
MilterSocket inet:32767@192.168.1.66
OnInfected Reject
PidFile /var/lib/clamav/clamav-milter.pid
ReportHostname mail.example.com
User vscan
VirusAction /usr/local/bin/virus_notify.sh
Text 4: Example clamav-milter.conf file (with comments removed)

Once the clamd scanning service is running and the freshclam service is maintaining the malware signatures the clamav-milter must be configured and started in order to connect the scanning service into Postfix's SMTP processing. The milter service is typically loads its configuration from the /etc/clamav-milter.conf file.
The service must be informed via the ClamdSocket directive where to find the clamd scanning service and via MilterSocket where to listen for connections from Postfix. The MitlerSocket directive is “inet:port@ip-address”. VirusAction and OnInfected directives can be used to control the behavior of the service when malware is identified; an OnInfected value of Quarantine will cause Postfix to hold the infected message in it's hold queue while a value of Reject will bounce the message with an SMTP error. Especially when used in Reject mode defining an appropriate VirusAction to notify the intended recipient of the message that a message has been discarded is important. The script named by VirusAction is executed in the security context of the scanning service and is provided seven parameters:
  1. Virus name-space
  2. Message queue id
  3. The sender's e-mail addres
  4. The e-mail address of the intended recipient.
  5. The subject of the message-id
  6. The message's Message-ID
  7. The date of the message.
Once configured the clamav-milter service must be started and set to automatically restart upon completion of system boot-up.
#!/bin/bash

# Parameters:
#   virus name, queue id, sender, destination, subject, message id, message date

(
 echo "";
 echo "   A message containing malware has been discarded.";
 echo "";
 echo "   Malware:     $1";
 echo "   Sender:      $3";
 echo "   Destination: $4";
 echo "   Subject:     $5";
 echo "   Message-ID:  $6";
 echo "   Date:        $7";
 echo "   Queue-ID:    $2";
 echo "";
) | \
 mail -s '[ALERT] Infected Messages Discarded' \
  -r milter@example.com -c helpdesk@example.com $4
Text 5: A sample script for use as the VirusAction. This script notifies the intended recipient and help-desk that a message was identified as malware and discarded.
Connecting the Postfix service to clamav-milter

In order to integrate the scanning into Postfix the milter is configured in the main.cf file as an smtpd_milter. The default action of the milter should be set to “accept” so if for any reason the milter is unresponsive messages will still be delivered. As when connecting the other components it is important to verify that the Postfix service can reach the specified service [traffic is permitted by firewall's etc...].
smtpd_milters = inet:milter.example.com:32767
milter_default_action = accept
Text 6: Configuration directives from Postfix's main.cf

Upon modification of the main.cf file the Postfix service should be restarted.
Once configured the malware filtering service should be tested; this can be accomplished by acquiring a copy of the EICAR diagnostic virus and verifying that messages with this content attached are rejected and that the end-user's are notified of the rejection [according the clamav-milter's defined VirusAction].

clamd[11973]: instream(127.0.0.1@60469): Eicar-Test-Signature FOUND
Text 7: Example clamd log message for identified malware.

When malware is detected a message will be logged by clamd via syslog regarding the event; this will typically be logged under the “mail” service. Depending on the distribution messages logged as mail will be written to either /var/log/mail or /var/log/maillog [at least with the default syslog configuration].

by whitemice (noreply@blogger.com) at February 06, 2012 06:38 AM

February 05, 2012

OpenGroupware (Legacy and Coils)

OIE XSLT Extension Methods

To facilitate the use of stylesheets as a component of workflow the OpenGroupware Integration Engine provides numerous functions callable as XSLT extensions. The extensions functions provide the ability to retrieve data from workflow messages into the style sheet as well as to access and manipulate sequences, exercise look-up tables, and search the server's database. These extension functions are available in the http://www.opengroupware.us/oie namespace.
<!-- Search to see if we can find a copy of that document-->
<xsl:variable name="documentid"
     select="oie:searchforobjectid(string('document'),
  'property.{http://example.com/financial}bankCode', string(bank_code),
  'property.{http://example.com/financial}documentType', 'PDF',
  'property.{http://example.com/financial}invoiceNumber', string(workorder),
  'property.{http://example.com/financial}invoiceDate', $invoicedate )"/>
<xsl:choose>  
  <xsl:when test="$documentid">
    <document_id isNull="false" dataType="integer"><xsl:value-of select="$documentid"/></document_id> 
  </xsl:when>
  <xsl:otherwise>  
    <document_id isNull="true" dataType="integer"/>    
  </xsl:otherwise>    
</xsl:choose>
Perform a search for the object id of the document matching the four specified conditions. In this example $invoicedate is a previously defined XSLT variable and bank_code a current element.
Currently implemented extension functions are:
  • sequencevalue(scope, name) - Retrieve the value of named sequence. An exception will occur if the sequence does not exist.
  • sequencereset(scope, name, value) - Set or reset the value of the named sequence to the specified value. This method will create a new sequence if a sequence by that name in the specified scope does not exist.
  • sequenceincrement(scope, name, increment) - Increment the named sequence by the specified value. An exception will occur if the sequence does not exist.
  • messagetext(label) - Retrieves the contents of the specified message within the current scope. If the messsage does not exist an exception will be raised by the underlying Logic operation. Care should be taken as to the encoding and type of data inserted into a style-sheet; no verification that the content of the message is suitable for inclusion in the style-sheet at the relevant point is performed.
  • searchforobjectid(domain, key, value, key, value, ….) - Performs a search of the specified domain using the provided key and value pairs. There is no hard limit on the number of value pairs that may be provided.  The domain must be one of: “appointment”, “contact”, “document”, “enterprise”, “process”, “project”, “resource”,  or “task” to identify the type of object being searched for.  If the search uniquely identities one entity of the specified type the object id of that entity is returned; if the search identifies either no entities or multiple entities an empty string is returned.  The intention of this method is to allow for content from the source document to be used to identity objects from the OpenGroupware database.
  • countobjects(domain, key, value, key, value, ….) - Performs a search of the specified domain using the provided key and value pairs. There is no hard limit on the number of value pairs that may be provided.  The domain must be one of: “appointment”, “contact”, “document”, “enterprise”, “process”, “project”, “resource”,  or “task” to identify the type of object being searched for.  The return value of the function is the number of objects that matched the search criteria up to 1,000. The search is limited to 1,000 results..
  • tablelookup(name, value) - Lookup the specified value in the named table.
  • reformatdate(value, format) - Reformats the StandardXML date or date-time value into the specified format. Since value is a string a length of 10 is assumed to represent a date input and a length of 19 to represent a date-time value.
  • datetimetodate(value) - Converts a StandardXML date-time representation to a StandardXML date representation.
  • stringtodate(value, format) - Reformats a date in the specified format to a StandardXML date representation.
  • stringtodatetime(value, format) - Reformats a date time in the specified format to a StandardXML date-time representation.
  • xattrvalue(name) - Returns the value of the named XATTR for the current process.  If no such XATTR is defined an empty string is returned.
All XSLT extensions are documented for the transform OIE action in the WMOGAG document. With the help of these methods XSLT transforms can be used to generate sophisticated and context-sensitive output documents.

by whitemice (noreply@blogger.com) at February 05, 2012 05:55 PM

Coils Development Update [2012-02-05]

This week multiple features and bug-fixes have been merged into the default branch.   OIE Table support has been notably improved and tested.  For the WebDAV presentation MKCOL (create folder), MOVE (renamed), and DELETE operations have been improved.
  • Cabinet projects can be created via WebDAV MKCOL operations in the Cabinets folder. The WebDAV folder /dav/Cabinets contains the root folders of each Project with a kind of "opengroupware.coils.cabinet" which the current user has access to. Creating a folder here actually creates a project of this kind whose root folder will be immediately discovered as a sub-folder.  A cabinet project and it's root folder both have a an object property which contains the cabinet name [a lower-case version of the folder name].
    • Tested in Nautilus/GVFS.
  • Cabinet projects can be deleted via WebDAV DELETE operations in the Cabinets folder.  
    • Tested in Nautilus/GVFS.
  • Folders in a Project Documents folder [via WebDAV MKCOL] can be created.  This includes the implementation of the folder::move Logic command.
  • Support for a defaultValue parameter added to StaticLookupTable.
  • Options added to SQLLookupTable for stripping and upper-casing table input and output values.  Documentation for this feature has already been added to WMOGAG.
  • Fixed a bug allowing an AdministrativeContext to operate in any specified security context; this allows administrative utilities to reduce their security contexts to operate in a diminished capacity much like setguid/setuid.
  • Implement Vista support for indexing Project entities.
  • Added Logic commands for dealing with company values
    • enterprise::get-companyvalue / set-companyvalue
    • contact::get-companyvalue / set-companyvalue
  • Close Issue#170 [ProjectAssignment.has_access issue, Legacy compatibility]

by whitemice (noreply@blogger.com) at February 05, 2012 05:07 PM

Building Legacy 5.5rc1

The following instructions provide a detailed step-by-step for how to build the OpenGroupware Legacy release candidate 1 on openSUSE 12.1.  Building on other distributions should be very similar. These instructions are intended for those interested in participating in OpenGroupware Legacy development and testing.

Step#1) Install dependencies
zypper in libapr-util1-devel libapr1-devel gcc46-objc libobjc46 postgresql-devel libmysqlclient-devel apache2-utils apache2-devel openldap2-devel libxmlsec1-devel libxmlsec1-gnutls-devel libxml2-devel libxslt-devel
Step #2) Get GNUstep
svn co http://svn.gna.org/svn/gnustep/modules/core
Step #3) Get SOPE
curl-o sope.tar.gz \
   http://www.sogo.nu/files/downloads/SOGo/Sources/SOPE-1.3.11.tar.gz
Step #4) Get OpenGroupware Legacy
hg clone http://opengroupware.hg.sourceforge.net:8000/hgroot/opengroupware/opengroupware
Step #5) Build and install GNUstep make
cd  core/make/
./configure
make
sudo make install
Step #6) Build and install GNUstep base
. /usr/local/share/GNUstep/Makefiles/GNUstep.sh
cd ../base
./configure --disable-tls
make
make check
sudo make install
Step #7) Build and install SOPE
cd ../..
tar xzvf sope.tar.gz
cd SOPE-1.3.11
./configure
make
sudo make install
sudo /sbin/ldconfig

Step #8) Build OGo
cd ../opengroupware/opengroupware/
hg pull
hg update
./configure
make APR=/usr/bin/apr-1-config APXS=/usr/sbin/apxs2
sudo make installsudo /sbin/ldconfig
Step #9) Configure OGo via defaults
If you have an existing OpenGroupware Legacy installation you probably already have the LSConnectionDictionary default defined appropriately. Note that the command used to editing defaults has changed from "Defaults" to "defaults".
su - ogo
mkdir /var/lib/opengroupware.org/run
defaults write ogo-webui WOPidFile /var/lib/opengroupware.org/run/ogo-webui.pid
defaults write ogo-zidestore WOPidFile /var/lib/opengroupware.org/run/ogo-zidestore.pid
defaults write ogo-xmlrpcd WOPidFile /var/lib/opengroupware.org/run/ogo-xmlrpcd.pid
defaults write ogo-webui WOLogFile /var/lib/opengroupware.org/run/ogo-webui.log
defaults write ogo-zidestore WOLogFile /var/lib/opengroupware.org/run/ogo-zidestore.log
defaults write ogo-xmlrpcd WOLogFile /var/lib/opengroupware.org/run/ogo-xmlrpcd.log
defaults write ogo-webui WOPort 20000
defaults write ogo-zidestore WOPort 21000
defaults write ogo-xmlrpcd WOPort 22000
defaults write NSGlobalDomain imap_host 127.0.0.1
defaults write NSGlobalDomain LSConnectionDictionary \
    '{userName="OGo"; databaseName="OGo";hostName="localhost";password="*******";}'
defaults write NSGlobalDomain NGBundlePath "/usr/local/lib64/opengroupware.org-5.5/commands:/usr/local/lib64/opengroupware.org-5.5/webui:/usr/local/lib64/opengroupware.org-5.5/datasources"
defaults write NSGlobalDomain LSModelName OpenGroupware.org_PostgreSQL
Step #9) Fire-up ZideStore
ogo-zidestore -WOLogFile - -WONoDetach YES -WOUseWatchDog NO
If it runs then your development build is probably OK!

by whitemice (noreply@blogger.com) at February 05, 2012 09:37 AM

February 02, 2012

OpenGroupware (Legacy and Coils)

OpenGroupware Legacy 5.5rc1

The OpenGroupware team is proud to announce the release of the first release candidate of OpenGroupware [Legacy] version 5.5. This OpenGroupware is a continuation of OpenGroupware.org 1.x [originally from Skyrix]. The source is now hosted as a SourceForge project.

It has undergone a long way and major changes to switch from old gnustep-make 1/libFoundation to gnustep-make 2 and gnustep-base. Further OpenGroupware now depends on the SOPE fork maintained by the SOGo developers at Inverse. A lot of changes went into OpenGroupware, and also SOPE to resolve compatibility and stability issues.

What is OpenGroupware
OpenGroupware is a web based collaboration platform and groupware solution:
  • LDAP or database authentication
  • manage contacts/companies
  • manage tasks
  • E-Mail Web Mail (IMAP client)
  • project and document management
  • GroupDAV interface
  • scripting Interface via zOGI
  • and more...

What's new
  • replaced dependency of gnustep-make V1.X with gnustep-make V2.X
  • replaced dependency of libFoundation with gnustep-base
  • now based on SOPE from the SOGo team
  • Experimental new features:
    • CTI with Asterisk
      • allows you to trigger calls via the Web Interface
    • Integration with location based services
      • geocoding of addresses using the Google Geocoding API
      • show list of contacts on a MAP
      • links to location based services on the Web for addresses
  • SNSD5 support dropped
  • We are working on how to get OGo multiple instances support back
Where to get it
You can download the tarball here: http://sourceforge.net/projects/opengroupware/files/

For installation instructions and information about required dependencies consult the INSTALL file in the tarball.

Where to get help
Best way is to ask for help on one of the mailing lists.

You can also meet an OpenGroupware developer at the FOSDEM 2012; there will be a talk about the new OGo release.

by whitemice (noreply@blogger.com) at February 02, 2012 07:31 AM

January 17, 2012

Whitemice Consulting

Whither Samba?

"What's up with Samba?" ....  "Is Samba4 for real?" ... "Is Samba4 ever going to be released?"  These questions have been common for several years now. Now we have answers thanks to this excellent article/interview on LWN.

by whitemice (noreply@blogger.com) at January 17, 2012 04:10 PM

January 11, 2012

Ben Rousch's Cluster of Bleep

Getting a Dropbox Public Link in KDE using Dolphin

I recently switched to Kubuntu and am falling in love with it, but I’ll talk more about that in a future post. This post is just about getting the Dropbox “Get public link” functionality working in Dolphin’s right-click menu. Under Gnome and Unity the default file manager in Nautilus so this functionality works after the standard nautilus-dropbox install, but KDE uses Dolphin as its file manager so we have to do a little bit of fiddling after the standard install to get it working.

I came across an old Dropbox forum post that has a simple way of adding the public link in the right-click menu. The final working version is scattered across a few posts and a blog, so I thought I’d pull it all together here for the rest of you migrating to KDE.

As usual, it’s pretty simple once you have it figured out. All you have to do is put a single file with the right commands in the right place. So open your favorite text editor and create the following file: ~/.kde/share/kde4/services/ServiceMenus/dropboxpublic.desktop . Then paste all of this stuff into it and save it:

[Desktop Entry]
Type=Service
ServiceTypes=KonqPopupMenu/Plugin
MimeType=application/octet-stream
Actions=CopyPublicLink
Encoding=UTF-8
Version=0.1

[Desktop Action CopyPublicLink]
Name=Get public link
Icon=klipper
Exec=dbus-send --type=method_call --dest=org.kde.klipper /klipper org.kde.klipper.klipper.setClipboardContents string:`dropbox puburl %f`

Now you should have a “Get public link” entry in the Actions sub-menu if you right-click a file in Dolphin. This will put the public URL in your clipboard so it’s ready to paste into your web browser, email client, IRC session, or whatever. Usually only files in the ~/Dropbox/Public directory have a public links, so if you use it on something outside of that directory nothing will be added to your clipboard.

P. S.
The original forum post is here: http://forums.dropbox.com/topic.php?id=12034&replies=2#post-76494
And the original blog post is here: http://ustunozgur.blogspot.com/2009/09/kde-submenu-action-for-getting-public.html

P. P. S.
If you’re not a Dropbox user, but you’d like to be, please use this Dropbox referral URL to sign up. If you do, you’ll get an extra 250MB of storage for free, and I’ll get and extra 500MB.Thanks!

by brousch at January 11, 2012 02:38 PM

January 03, 2012

Whitemice Consulting

Streaming the BBC World Service

Every leap year the american public radio service becomes obsessed with providing non-stop glossy and shallow coverage of the various presidential candidates, campaigns, and caucases.  They report endlessly on various polling results [even though the validity of such polls is in question according to credible statisticians and demographers].  Fortunately there is the BBC World Service which offers an alternative for english speaking listeners exhausted by caucus mania.  The appropriate links are not easy to find - but they are there. The MP3 link under "World Service English (Internet Schedule)" works perfectly when added to Banshee's radio playlist.

by whitemice (noreply@blogger.com) at January 03, 2012 07:37 AM

December 29, 2011

OpenGroupware (Legacy and Coils)

Introducing Vista Search

The first version of the Vista Logic and Protocol bundles have been merged into the default branch.  Vista provides a fast full-text indexing method for content stored on the OpenGroupware Coils server.  The Vista component can currently index Contact, Document [meta-data], Enterprise, Note, and Task entities.  All the text fields of these entities are indexed including the values of object properties and company values. Utilizing the power of Vista search is performed by performing HTTP requests to the protocol exposed at "/vista" on the server. 
curl -v -u fred -o output 'http://coils.example.com/vista?term=detroit&term=steel&archived&type=enterprise'
Authenticate as user fred and search for enterprise entities, regardless of archived status, that contain the terms "detroit" and "steel".
The results of the HTTP request are JSON encoded Omphalos representations of the first 100 entities to match the specified criteria.  The default Omphalos detail level is 2056 [Comment + Company Values].  If an alternate detail level is desired this default can be overridden using the "detail" URL parameter.   The following URL parameters are supported:
  • archived - Include entities in the search regardless of there archived status. If no specified archived entities will be excluded from the search results.
  • detail - The Omphalos detail level to use when representing entities in the response.  It is important to recognize that specifying a high detail level will reduce performance.
  • term - Specify a search term.  Any number of terms may be specified.
  • type - Limit the searched entities by type.  If no type is specified all indexed entities are searched regardless of type.  Multiple type parameters may be specified.
For code local to the server searches can be performed using the "vista::search" Logic command:
results = ctx.run_command('vista::search', keywords = [ 'detroit', 'steel' ],
                                                                    entity_types = [ 'enterprise' ],
                                                                    include_archived = True)
Peform a Vista search via Logic for all enterprises, regardless of archived status, that contain the terms "detroit" and "steel".

The recently packaged tool coils-request-index can request the creation or update of an entities search vector.  Normally when an entity is modified a re-index is requested automatically [search vector generation happens in the background and is performed by the coils.vista.index component].  However, if large changes are made to the database or for the initial index generation the use of coils-request-index may expedite the process.
coils-request-index --contact --enterprises --notes --documents --tasks
Request an index/reindex of all the entities of the specified types.
coils-request-index --objectid=10100
Request an index/reindex the entity with objectId 10,100.
If the index is already current for the entity the vector generation request will be discarded.
This new feature does require a schema update to existing OpenGroupware databases.  This schema update will be required for version 0.1.45.
CREATE TABLE vista_vector (
  object_id  INT PRIMARY KEY,
  version    INT DEFAULT 0,
  edition    INT,
  entity     VARCHAR(25) NOT NULL,
  event_date DATE DEFAULT 'TODAY', 
  archived   BOOL DEFAULT FALSE,
  keywords   VARCHAR(128)[],
  vector     tsvector);
CREATE INDEX vista_idx_i0 ON vista_vector (entity);
CREATE INDEX vista_idx_i1 ON vista_vector (event_date);
CREATE INDEX vista_idx_i2 ON vista_vector USING gin(vector);
Vista search utilizes PostgreSQL's powerful tsearch text indexing module.  tsearch provides lexeme oriented indexing - so the server knows, for example, that "rats" and "rat" share the same stem.  Thanks to tsearch Vista searches are not only fast - they're clever!

by whitemice (noreply@blogger.com) at December 29, 2011 07:51 AM

December 28, 2011

OpenGroupware (Legacy and Coils)

Accessing Server Configuration (Defaults)

When developing Logic, either a Command or a Service component, one frequent need is to check the value of a server's configuration directive [a "default" in OpenGroupware speak].  Accessing server configuration is performed using an instance of the ServerDefaultsManager object.  The following code retrieves the value of the CoilsListenAddress address; and if no such default is defined it returns the value "127.0.0.1".
sd = ServerDefaultsManager()
HTTP_HOST = sd.string_for_default('CoilsListenAddress', '127.0.0.1')
The ServerDefaultsManager will cache the server's configuration - so if you are going to be checking a lot of defaults is better to keep the object around rather than repeatedly creating it. The ServerDefaultsManager provides the following methods for retrieving server defaults:
  • bool_for_default(directive) - Values of boolean configuration values are stored as "YES" and "NO" strings.  Actually, any value that isn't "YES" is interpreted as False, which is also the default if no such directive is defined. The value returned by the method is a Python bool type.
  • string_for_default(directive, default value) - Returns the value of the default as string or returns the specified default value if no such directive is defined.
  • integer_for_default(directive, default value) - Returns the value as an integer, or returns the specified default value if no such directive is defined. An exception is raised if the value cannot be represented as an integer.
  • default_as_dict(directive, default value) - Returns the value of the specified directive as a dictionary, or the specified default value if no such directive is defined.  An exception is raised if the value is not a dictionary.
  • default_as_list(directive, default) - Returns the value as a list, or the specified default value if no such directive is defined.  An exception is raised if the value is not a list.
Regarding the actually loading of defaults the defaults manager will load from (or save to) one of two sources.  If the file ".server_defaults.pickle" exists in the document root of the server the defaults are loaded from (and saved to) that Python pickle file; otherwise the defaults are loaded from (and saved to) the OpenSTEP plist file at ".libFoundation/Defaults/NSGlobalDomain.plist".  Use of the OpenSTEP plist file facilitates parallel operation of OpenGroupware Coils with Legacy - both OpenGroupware Coils and OpenGroupware Legacy will operate using the shared configuration. 
One caveat to remember is that OpenSTEP plist files are always stored in the ISO8859-1 encoding.  This includes both the server defaults and user defaults.  Both OpenGroupware Coils and OpenGroupware Legacy always store user defaults in OpenSTEP plist format.  Facilities for parsing and writing OpenSTEP plist files are provided by the coils.foundation module.
If you are developing a remote component that does not have access to the server's document root your component can acquire a copy of the server's configuration by sending a "get_server_defaults" message to the coils.administrator component.  The payload of the response should contain the cluster's GUID [as the "GUID" key] and a copy of all the server defaults [in the "defaults" key).

by whitemice (noreply@blogger.com) at December 28, 2011 08:44 PM

December 13, 2011

Whitemice Consulting

Really? A SHMMAX of 36MB

I was running some tests on OpenGroupware Coils on my new HP workstation - and PostgresSQL seemed to be huffing-and-puffing like an overweight guy trying to run up stairs.  Huh.  What is the first thing a PostgreSQL administraor always checks? The shared_buffers parameter [in /var/lib/pgsql/data/postgresql.conf]; the default package typically sets this value to some absurdly small value resulting in dreadful performance.  This default value is the principal reason the absurd notion that "MySQL is faster than PostgreSQL" got traction.  Sure enough - the default value is 24MB! Yikes.  So I changed that setting to a more reasonable 512MB.
shared_buffers = 512MB
Then I tried to restart PostgreSQL, and it flopped.
$ service postgresql start
redirecting to systemctl
Job failed. See system logs and 'systemctl status' for details.
Looking in /var/log/messages I see:
Dec  9 19:27:23 workstation postgresql[7288]: Starting PostgreSQL2011-12-09 19:27:23 EST   FATAL:  could not create shared memory segment: Invalid argument
Dec  9 19:27:23 workstation postgresql[7288]: 2011-12-09 19:27:23 EST   DETAIL:  Failed system call was shmget(key=5432001, size=76685312, 03600).
Huh.  I remember an error message like that from the RedHat 6.x & 7.x days when the kernel's default settings regarding System V IPC resources where really low.  Back then you possibly had to go look at defines in header files for these values. Fortunately we now have sysctl.  And what does sysctl tell us?
$ sysctl kernel.shmmax
kernel.shmmax = 33554432
What?!?!  That value is in bytes - so the limit is ~32MB?!  This is on a 64-bit installation with 8GB of RAM. That makes no sense at all.  Time to change that to something reasonable:
$ sysctl -w kernel.shmmax=1000000000
And now "service postgresql start" succeeds.  The ipcs command which reports System V IPC resources allocated in the system shows that the PostgreSQL engine has allocated the expected buffer pool:
$ ipcs
------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status     
0x0052e2c1 1212416    postgres   600        572383232  4  
Once you change a sysctl setting in a helpful way the change has to be added to the /etc/sysctl.d/ so that it gets re-applied when the machine reboots;  otherwise our PostgreSQL instance is going to fail to start next time.  Create a file in that directory with a name such as postgresql.conf containing a single line of -
kernel.shmmax = 1000000000
The files in /etc/sysctl.d/ are processed after distribution and package defaults are applied.  It is also possible to edit /etc/sysctl.conf; however manual changes to that file may get overwritten by system management tools.  So use the /etc/sysctl.d/ strategy to override distribution defaults.
Now, back to work.

by whitemice (noreply@blogger.com) at December 13, 2011 08:37 PM

December 12, 2011

Whitemice Consulting

Querying Connectivity

You're application almost always needs to know if there is a working network connection.  This is typically handled by placing the connection attempt in a try...catch block.  That works, but can be slow, and it means the UI can't really adapt to the level of current connectivity.  A much better solution is to query the NetworkManager [used by every mainstream distribution] via the System D-Bus for the current connectivity.  This method is used by many applications from GNOME's Evolution to Mozilla's Firefox - but it doesn't seem to get much press coverage.  So here is a simple example to query connectivity via Python [assuming NetworkManager 0.9 or later]:

#!/usr/bin/env python
import dbus

NM_BUS_NAME       = 'org.freedesktop.NetworkManager'
NM_OBJECT_PATH    = '/org/freedesktop/NetworkManager'
NM_INTERFACE_NAME = 'org.freedesktop.NetworkManager'
NM_STATE_INDEX = {  0: 'Unknown',
                   10: 'Asleep',
                   20: 'Disconnected',
                   30: 'Disconnecting',
                   40: 'Connecting',
                   50: 'Connected (Local)',
                   60: 'Connected (Site)',
                   70: 'Connected (Global)' }

if __name__ == "__main__":
    bus = dbus.SystemBus()
    manager   = bus.get_object(NM_BUS_NAME, NM_OBJECT_PATH)
    interface = dbus.Interface(manager, NM_INTERFACE_NAME)

    state = interface.state()
    if state in NM_STATE_INDEX:
        print('Current Network State: {0}'.format(NM_STATE_INDEX[state]))
    else:
        print('Network Manager state not recognized.')
FYI: if you search the interwebz for the NetworkManager API specification ... every search engine will send you to the wrong place; either just wrong or to the documentation of an older version of the API. The current API specification is here.

by whitemice (noreply@blogger.com) at December 12, 2011 07:00 AM

GNOME3 Journal Extension

Now that's what I'm talking about!  A new extension just showed up on extensions.gnome.org that adds a "Journal" tab to the already awesome GNOME3 overview.  It integrates with Zeitgeist to provide access to recently or heavily used categories of items - sort of like "Recent" but all grown up and with college smarts.  And installing it is as easy as clicking "On" [assuming you have Zeitgeist already installed].
Journal tab in Overview
 A very handy addition that adds to the same concept provided by the gnome-activity-journal [which is packaged for openSUSE, BTW].

by whitemice (noreply@blogger.com) at December 12, 2011 06:12 AM

December 09, 2011

Whitemice Consulting

"Secret" extensions.gnome.org links

The new extensions.gnome.org is a fabulous idea and well-executed concept. But one issue is that is it isn't easy to visualize what is new; and the content of the site is growing rapidly.  As it turns out there is a secret link to sort extensions by the order they were added: https://extensions.gnome.org/?sort=recent For all the RSS user's out there the site also provides an RSS feed: https://extensions.gnome.org/rss/ Enjoy!
Note: Seriously people - this site is barely two weeks old.  Have some patience. It seems doubtless that the obviously inspired developers will add the "obviously" needed features in due time

by whitemice (noreply@blogger.com) at December 09, 2011 06:12 AM

December 07, 2011

Whitemice Consulting

TortoiseHg Packages

It turns out some kind fellow has packaged TortoiseHg for openSUSE 12.1;  you can find the packages in the home:tzotsos:vcs repo.  The repo provides not just the Tortoise GUI but the nautilus extensions as well. The only item the packages seem not to provide is a .desktop file for starting the thg application - without a .desktop file GNOME3 doesn't know the application exists.  You can either create the required file using alacarte or by hand; if by hand open the [new] file at ~/.local/share/applications/tortoise.desktop and paste in:
#!/usr/bin/env xdg-open

[Desktop Entry]
Version=1.0
Type=Application
Terminal=false
Icon[en_US]=/usr/share/pixmaps/tortoisehg/icons/thg_logo_92x50.png
Exec=/usr/bin/thg
Name[en_US]=Tortoise Hg
Comment[en_US]=Mercurial Client
Name=Tortoise Hg
Comment=Mercurial Client
Icon=/usr/share/pixmaps/tortoisehg/icons/thg_logo_92x50.png
Now GNOME3 will know about the thg application and you can launch it via the application search as well as add it to your launcher.

by whitemice (noreply@blogger.com) at December 07, 2011 07:11 AM

December 05, 2011

Whitemice Consulting

Enabling the RabbitMQ Management Plugin

Prior to 2.7.x version of RabbitMQ it was necessary to manually install the plug-ins that provided the management interface [as well as their dependencies]. Now in the 2.7.x series the management interface plug-in and related dependencies are included - but not enabled by default.  The management plug-in must be toggled into the enabled state using the new rabbitmq-plugins command.  Enabling a plug-in will automatically enable any other plug-ins that the specified plug-in depends on Whenever you enable or disable a plug-in you must restart the sever.
If you have a brand new 2.7.x instance installed, turn on the plug-in with:
service rabbitmq-server stop
rabbitmq-plugins enable rabbitmq_management
service rabbitmq-server restart
When you performed the rabbitmq-plugins command you should have seen the following output:

The following plugins have been enabled:
  mochiweb
  webmachine
  rabbitmq_mochiweb
  amqp_client
  rabbitmq_management_agent
  rabbitmq_management
You management interface at TCP/55672 should be available.  The initial login and password are "guest" and "guest".  You want to change those.

by whitemice (noreply@blogger.com) at December 05, 2011 07:48 AM

December 01, 2011

Whitemice Consulting

Using gedit to make a list of values into a set.

gedit is awesome;  the flexibility of the tool continues to impress me.  One problem I'm frequently faced with is a list of id values from some query, or utility, or e-mail message... and I want to do something with them.  So, for example I have:
10731
10732
10733
10734
10735
10736
10737
10738
10739
but what I need is those id values as a sequence such as for use in an SQL IN expression or to assign to a Python set or list.  What I want is:
(10731,'10732', '10733', '10734', '10735', '10736', '10737',
'10738', '10739')
Reformatting a few numbers by hand isn't too hard - but what if I have a list of hundreds of id values? The answer, of course, is provided by gedit.  Under Tools -> Manage External Tools the user can build filters that can be applied to documents and have the results returned to gedit.  If I create a new external tool that accepts as input the "Current document" and as output has "Replace current document" then gedit will replace the contents of the current document with the results of the filter [pretty obvious;  and if it doesn't work I can always Ctrl-Z].  The body of the filter can be any script - a Python script is perfectly valid. Like this hack:
#!/usr/bin/env python
import sys

iteration = 0
line_length = 0
text = sys.stdin.readline()
while (text !=  ''):
  text = text.strip()
  if (len(text) > 0):
    if (iteration == 0):
      sys.stdout.write('(')
    else:
      sys.stdout.write(', ')
    if (line_length > 74):
      sys.stdout.write('\n ')
      line_length = 0
    if (len(text) > 0):
      sys.stdout.write('\'{0}\''.format(text))
    line_length = line_length + len(text) + 4
    iteration = iteration + 1
  text = sys.stdin.readline()
sys.stdout.write(')') 
sys.stdout.flush()
The current document becomes the standard-input for the script and the standard-output of the script will replace the current document. The above hack reads in a list of lines and returns them as a set enumeration nicely wrapped to 80 characters per line. External tools are saved under names; for this one I saved it as "IN-Clause-Filter".
Now that I've setup the external tool every time I paste a list of id values into gedit I can simply select Tools -> External Tools -> IN-Clause-Filter and my list is instantly turned into a set enumeration.

by whitemice (noreply@blogger.com) at December 01, 2011 09:00 AM

November 28, 2011

Whitemice Consulting

Samba-VirusFilter 0.1.3 Released

Samba-VirusFilter (the heir to samba-vscan) version 0.1.3 was released a few days ago [download].  Samba-VirusFilter supports ClamAV, F-Secure, and Sophos.  A sample ClamAV configuration snippet for your smb.conf looks like:
vfs objects = svf-clamav
svf-clamav:scan on open = yes | no
svf-clamav:scan on close = yes | no
svf-clamav:max file size = (bytes, default 10^8)
svf-clamav:min file size = (bytes, default 10)
svf-clamav:infected file action = nothing | quarantine | delete
svf-clamav:quarantine directory  = directory
svf-clamav:quarantine prefix = string
svf-clamav:infected file command = command
svf-clamav:scan error command = command
If you use a quarantine directory don't forget to make sure the Samba daemon will always have sufficient access to put files in there.

by whitemice (noreply@blogger.com) at November 28, 2011 05:15 AM

November 27, 2011

Whitemice Consulting

Overlooked Content From 2011

Yes, it is almost 2012.
Looking back for overlooked content I'm compelled to mention the 2011 annual "LDAPCon" conference focused on directory services and LDAP; the organizers/hosts have made available some excellent videos and papers [index page]. These are an unrivaled source of information on the current state and future of directory services.
The same is also true of the annual SambaXP conference in relation to all things Samba and CIFS [2011 content index].
Podcast content in general tends to be lite and aimless fare;  those sources provide some meat for the meal.

by whitemice (noreply@blogger.com) at November 27, 2011 10:59 AM

All those SQLite databases...

Many current application use the SQLite database for tracking information; this includes F-Spot, Banshee, Hamster, Evolution, and others.  Even the WebKit component uses SQLite [you might be surprised to discover ~/.local/share/webkit/databases].  It is wonderfully efficient that there is one common local data storage technique all these applications can use,  especially since it is one that is managable using a universally known dialect [SQL]. But there is a dark-side to SQLite.  Much like old Dbase databases it needs to be vacuumed.  And how reliably are all those applications providing their little databases with the required affection?  Also, do you trust those lazy developers to have dealt with the condition of a corrupted database?   If an application hangs, or is slow, or doesn't open... maybe that little database is corrupted?
Aside: As a system administrator for almost two decades I do not trust developers. They still put error messages in applications like "File not found!".  Argh!
On the other hand SQLite provides a handy means of performing an integrity check on databases - the "PRAGMA integrity_check" command.  I've watched a few of these little databases and discovered that (a) they aren't often all that little, and (b) manually performing a VACUUM may dramatically reduce their on-disk size.  Both these facts indicate that developers are lazy and should not be trusted.
Note: in at least one of these cases the application has subsequently been improved. Developers do respond rather quickly when offered a blend of compliments spiced with bug reports.  No, I'm not going to name offending applications as that is too easily used as fodder by nattering nabobs.  And even the laziest Open Source developer is working harder than their proprietary brothers.
In light of this situation my solution is a hack - a Python script [download] that crawls around looking for SQLite databases.  First the script attempts to open the database in exclusive mode, then it performs an integrity check, and if that succeeds it performs a vacuum operation.  Currently it looks for databases in "~/.local/share" [where it will find databases managed by application appropriately following the XDG specification], "~/.cache", "~/.pki", "~/.rcc", and "~/.config".
Download the script and run it. Worst thing that happens is that it accomplishes nothing.  On the other hand it might recover some disk space, improve application performance, or reveal a busted database.

by whitemice (noreply@blogger.com) at November 27, 2011 09:11 AM

November 25, 2011

Whitemice Consulting

gnome-tweak-tool

GNOME3 simplified many things.  In the process some settings and preferences got removed from the primary user interface.  A side-effect of that is a fair number of BLOG posts like "Dude where’s my settings?".  Many of these BLOG posts are informative; they explain how to get to the preference value via dconf either by the command line gsettings tool or the GUI dconf-editor [which replaces GNOME2's gconf-editor].  Both of those are good methods. Knowing how to use gsettings in case of an epic-fail situation is a very useful skill to have. On the other hand - most of the settings discussed in these posts can be easily tweaked using the appropriately named gnome-tweak-toolgnome-tweak-tool provides a friendly GUI to a variety of maybe-you-shouldn't-mess-with-this-but-here-we-are kind of preferences.  The gnome-tweak-tool package is available in the standard repositories for openSUSE 12.1

by whitemice (noreply@blogger.com) at November 25, 2011 08:54 AM

GAJ, Zeitgeist, & openSUSE 12.1

In openSUSE 12.1 the GNOME Activity Journal and Zeitgeist data hub are only a package install away -
zypper in gnome-activity-journal
Now the GNOME Activity Journal is available;  an excellent productivity tool.  Hopefully more data providers will appear soon.

by whitemice (noreply@blogger.com) at November 25, 2011 06:57 AM

Ctrl-Alt-Shift-V ... Pasting Happiness

If you cut or copy text from an application [especially a web browser] and then paste it into LibreOffice what you often get is formatted text, or at least some approximation of the text's original formatting.  This is awful.  When using LibreOffice appropriately all formatting is managed via the excellent support for styles. All you want is the text - and nothing about the text.  I've used the brute force solution in the past of bouncing my cut-n-paste through gedit.  Until I discovered Ctrl-Alt-Shift-V.  Yes, all four keys at once.  Ctrl-Alt-Shift-V is un-formatted paste [it pastes nothing but the text].  Pasting happiness!

by whitemice (noreply@blogger.com) at November 25, 2011 06:25 AM

November 23, 2011

Whitemice Consulting

TortoiseHg & openSUSE 12.1

TortoiseHg is an excellent GUI for working with code from a Mercurial [aka Hg] repository.  You can just download the source and run "./thg". The previous version of TortoiseHg used PyGTK and pretty much just-worked.  Recently they switched to using Qt4.  Only once you install Qt4 support for Python the GUI still doesn't run - it fails with an error:
ImportError: No module named Qsci
What is "Qsci"?  "Qsci" is short hand for Qscintilla which is a Qt port of the Scintilla editing component.  This is provided to Python as the PyQt4.Qsci module.  This module is installed separately from the main Qt4 module in a package called "python-qscintilla" [notice a pattern where this feature of Qt4 is referred to by at least three names?].  Once you've run down this quirky daisy-chain of names it is just:
zypper in python-qscintilla
and now thg runs.

by whitemice (noreply@blogger.com) at November 23, 2011 06:59 AM

November 21, 2011

Ben Rousch's Cluster of Bleep

A Few CyanogenMod 7.1 on Nook Color Fixes

I’m currently running CyanogenMod 7.1 on my Barnes & Noble Nook Color and I absolutely love it. I feel like I’m living in the future when I use this thing. However it’s not perfect, and I have rectified a couple of annoyances that I should document here for anyone else with these problems who should wander by.

Old Android Market installed

I’m running CyanogenMod 7.1 on both my Droid phone and my Nook Color tablet, but for some reason one of them had an old version of the Android Market. Unfortunately I can’t remember now which one was stuck on an old version. I ended up slogging through the interwebs until I found a working up-to-date APK for the Android Market. I’ve stashed a copy of the Android Market 3.1.5 APK in my Public Dropbox for anyone else who needs it.

Unable to install Google Earth app

Google Earth on Nook Color
For some reason the Android Market said I could not install the Google Earth app. I have no idea why. I ended up scouring the backwaters XDA and came upon an APK for it. This installed and is working well for me. I have placed a copy of the Google Earth APK in my public Dropbox for anyone else who needs it.

Weirdness in the Nook for Android app

Broken Nook for Android MagazinesWorking Nook for Android Magazines
I subscribed to Popular Science and National Geographic magazines while I was using the standard Nook Color operating system. After my upgrade to CM7.1 I could only download the first issue of each magazine because they all appeared stacked on top of each other. I finally stumbled on this CyanogenMod forum post which offered a few possible solutions.

To save you some reading, it looks like CM7.1 sets the Nook Color screen’s DPI to 160. The Nook for Android app assumes that the density is more than 160DPI, and this apparently causes the problem with the magazines stacking. I installed the free LCD Resolution app from the Android Market and used to set my resolution back to the default 169DPI. This fixed the problem.

by brousch at November 21, 2011 09:44 PM

November 01, 2011

Whitemice Consulting

Converting M4B's to MP3

I ended up with some M4B audio files; these are "MPEG v4 system, iTunes AAC-LC" files.  In order to reliably manage these files along with every other audio file [all of which are MP3] the simplest solution is just to convert them to MP3.  In order to accomplish that I dumped them back out to WAV using mplayer and re-encoded them to MP3 using lame.  Both lame and mplayer are available for openSUSE from the PacMan repositories, so you can easily install them via zypper.

mplayer "filename.m4b" -nojoystick -ao pcm:file=tmp.wav
lame -b 128 -q1 tmp.wav tmp.mp3

The "-nojoystick" option for mplayer isn't required but it prevents mountains of output about mplayer being unable to read the joystick device [most likely due to the fact that I don't have a joystick].  I left the bit-rate for lame at 128 since there is no point in re-encoding a file at a higher bit-rate than the originally encoded file - these files are mono-channel human speech not high-fidelity audio.

by whitemice (noreply@blogger.com) at November 01, 2011 07:15 PM

"samba-vscan" Is Dead, Long Live "samba-virusfilter"!

Noticed an interesting message on the Samba-Technical list today.  The Samba VFS module "samba-vscan" which has long been used to build integrated malware detection into Samba is no longer supported for the 3.6.x series.  SATOH Fumiyasu has been patching samba-vfs for 3.3 and up through 3.5; but with 3.6.1 he has created a new module named samba-virusfilter.  His message is here.

by whitemice (noreply@blogger.com) at November 01, 2011 06:15 AM

October 31, 2011

Whitemice Consulting

Where and what is /var/run/named?

# service named start
Starting name server BIND checkproc: Can not stat /var/run/named/named.pid: Too many levels of symbolic links
- Warning: /var/run/named/named.pid exists! start_daemon: Can not stat /var/run/named/named.pid: Too many levels of symbolic links
                                                                     done

Eh?  Somehow I messed up wherever /var/run/named is supposed to be.  This happened when changing a root-jail DNS server to a non-jailed server.  After toggling the NAMED_RUN_CHROOTED value to "no" in /etc/sysconfig/named starting named complains about this [this named is meant to integrate with Samba4]. Seems strange.  Once you try to restart named after this change /var/run/named is automatically created as a directory - but it doesn't work.  This fix is to stop named and create the correct symbolic link:
ln -s /var/lib/named/var/run/named /var/run/named
Not sure how this situation happens; but now the fix/gotcha is here for the search engines to crawl.

by whitemice (noreply@blogger.com) at October 31, 2011 02:05 PM

Reformatting an iPod

I have an iPod whose content seems to have gone wonky.  Delete's fail, play-lists won't sync, etc... I wanted to start-over.  Unfortunately the iPod itself doesn't have any useful "reset" feature.  According to the interwebs you need Apple's iTunes application in order to reformat an iPod.  Alright, time to find another solution; and the winner is:

mkfs.vfat -F 32 -I -n "iPod Name" /dev/sdb1
A good old-school reformat. After reformatting and resetting [hold down select and play for 15 seconds] the device is back to its original state.

by whitemice (noreply@blogger.com) at October 31, 2011 07:59 AM

Implementing Queue Expiration w/RabbitMQ

The latest versions of RabbitMQ support a feature where idle queues can be automatically deleted from the server.  For queues used in an RPC or workflow model this can save a lot of grief - as the consumers for these queues typically vanish leaving the queue behind. Over time these unused queues accumulate and consume resources on the server(s). If you are using pyamqplib setting the expiration on a queue is as simple as:

import amqplib.client_0_8 as amq
connection = amq.Connection(host="localhost:5672", userid=*, password=*, virtual_host="/", insist=False)
channel = connection.channel()
queue = channel.queue_declare(queue="testQueue", durable=True, exclusive=False, auto_delete=False, arguments={'x-expires': 9000})
channel.exchange_declare(exchange='testExchange', type="fanout", durable=False, auto_delete=False)
channel.queue_bind(queue="testQueue", exchange='exchange')

Now if that queue goes unused for 9 seconds it will be dropped by the server [the value is in milliseconds]. So long as the queue has consumers it will persist, but once the last consumer has disconnected and no further operations have occurred - poof, you get your resources back.

by whitemice (noreply@blogger.com) at October 31, 2011 07:47 AM

October 26, 2011

OpenGroupware (Legacy and Coils)

Invoking an OIE Route from PHP

The repository not contains a PHP class making it simple to invoke an OIE workflow from PHP.  See the oie.php file.  Using the OIEProcess class defined in the file processes can be created and the process id and input message UUID known.

$HTTPROOT   = "http://coils.example.com";
$ROUTENAME  = "TEST_MailBack";
$PARAMETERS = array('myParameter'=>'YOYO MAMA', 'otherParam'=>4);
$request = new OIEProcess($HTTPROOT, $ROUTENAME, $PARAMETERS);
if ($request->start('adam', '*******', fopen('/etc/passwd', 'r'), 'text/plain') == 'OK') {
    echo "\n";
    echo "Process ID: " . $request->get_process_id() . "\n";
    echo "Message UUID: " . $request->get_message_id() . "\n";    
}

The start method returns either "OK", "OIEERR" (OIE refused the request), or "SYSERR" (the curl operation failed).  The first and second parameters for start is the user credentials, the optional third and fourth parameters is the input message stream and the payload mimetype.  If no mimetype is specified a default of "application/octet-stream" is assumed.

by whitemice (noreply@blogger.com) at October 26, 2011 08:35 AM

October 24, 2011

OpenGroupware (Legacy and Coils)

Installing Horde On CentOS6

Progress is being made on first-class integration between OpenGroupware Coils and Horde 4; that is, using Horde 4 as a Web 2.0 / AJAX front-end to the various services provided by OpenGroupware Coils.  This integration is primarily implemented using a custom JSON-RPC protocol bundle designed specifically for integration with Horde.  This article walks through the install to achieve a basic Horde installation.  Subsequent articles will document how to achieve OpenGroupware Coils integration.

This installation procedure assumes:
  • You'll be using a memcache instance for caching.
  • A PostgreSQL database for server meta-data and Horde user preferences;  probably the same PostgreSQL instance you use for your OpenGroupware databse. But for this example we are just setting up a local PostgreSQL instance.  We'll cover installing OpenGroupware Coils on CentOS6 soon.
  • You'll be installing Horde into a virtual host in the folder "/srv/www/vhosts/horde".
  • The horde install will have it's own PEAR repository as separated from the system PEAR repository as possible.
  • The ImageMagick packages will be installed to allow Horde to manipulate images (such as creating thumbnails of image attachments to e-mail).
  • GPG will be installed in order to support encrypted e-mails and notes.
  • We are starting with a clean CentOS6 install of the basic server profile.
  • SELinux is disabled (edit /etc/sysconfig/selinux). In a subsequent article re-enabling SELinux will be documented. 
  • To access this instance remotely the TCP/80 port must be allowed via the host's firewall configuration. If you intend to enable TLS/SSL (secure) access port TCP/443 must also be allowed [on CentOS6 use the system-config-firewall-tui to perform basic firewall configuration configuration, production systems should consider using a more sophisticated tool such as FWBuilder].
Step#1 Install the required packages.
yum install php-devel php-pear make gcc libidn-devel pam-devel pcre-devel postgresql-devel libidn-devel memcached-devel memcached libmemcached zlib-devel cyrus-sasl-devel ImageMagick-devel ImageMagick php-ldap php-intl php-mbstring php-pdo php-pecl-apc php-pgsql php-soap php-tidy php-xml php-xmlrpc php-pecl-memcache libtidy-devel
Step#2 Create the vhost directory and initialize the PEAR package database.
mkdir -p /srv/www/vhosts/horde
pear config-create  /srv/www/vhosts/horde /srv/www/vhosts/horde/pear.conf
pear -c /srv/www/vhosts/horde/pear.conf install pear
Step#3 Add the Horde channel to the PEAR configuration and initialize the Horde role.  The last "run-scripts" command will prompt you for the root of the Horde installation; enter "/srv/www/vhosts/horde".
/srv/www/vhosts/horde/pear/pear -c  /srv/www/vhosts/horde/pear.conf channel-discover pear.horde.org
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf  install horde/horde_role
/srv/www/vhosts/horde/pear/pear -c  /srv/www/vhosts/horde/pear.conf run-scripts horde/Horde_Role
Step#4 Set the timezone in your php.ini file Edit the /etc/php.ini to set the date.timezone property to the server's local timezone.  For example: "date.timezone=America/Detroit"

Step#5 Install the re2c package from the DAG repo.  You can optionally add the DAG repo to your system or just pull this one package.  re2c is used by the PHP interpreter to efficiently compile regular expressions.
curl --location -o /tmp/re2c-0.13.5-1.el6.rf.x86_64.rpm http://mandril.creatis.insa-lyon.fr/linux/dag/redhat/el6/en/x86_64/dag/RPMS/re2c-0.13.5-1.el6.rf.x86_64.rpm
rpm -Uvh /tmp/re2c-0.13.5-1.el6.rf.x86_64.rpm
Step#6 As in Step#5 you can add the DAG repo to your system or just pull the two packages necessary to build the geoip module.  Horde will use this to relate hosts to geographic regions.
curl --location -o /tmp/geoip-devel-1.4.6-1.el6.rf.x86_64.rpm http://mandril.creatis.insa-lyon.fr/linux/dag/redhat/el6/en/x86_64/dag/RPMS/geoip-devel-1.4.6-1.el6.rf.x86_64.rpm
curl --location -o /tmp/geoip-1.4.6-1.el6.rf.x86_64.rpm http://mandril.creatis.insa-lyon.fr/linux/dag/redhat/el6/en/x86_64/dag/RPMS/geoip-1.4.6-1.el6.rf.x86_64.rpm
rpm -Uvh  /tmp/geoip-1.4.6-1.el6.rf.x86_64.rpm /tmp/geoip-devel-1.4.6-1.el6.rf.x86_64.rpm
pecl install geoip
echo "extension=geoip.so" > /etc/php.d/geoip.ini
Step#7 Build and install the Imagick extension which will allow Horde to efficiently manipulate images.
pecl install Imagick
echo "extension=imagick.so" > /etc/php.d/imagick.ini
Step#8 Build the tidy module which Horde can use to sanitize HTML content.
pecl install tidy
echo "extension=tidy.so" > /etc/php.d/tidy.ini
Step#9 Build the lzf module which allows Horde to efficiently compress and decompress data.
pecl install lzf
echo "extension=lzf.so" > /etc/php.d/lzf.ini
Step#10 Install the PEAR packages. In this example we manually install several PEAR modules first to verify that PEAR installation is working and because we want to ensure that these optional modules get installed as we will be depending on their existence in this setup.  Particularly the Net_Sieve and Horde_Memcache do not install my default./srv/www/vhosts.
/srv/www/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install HTTP_Request
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install Net_SMTP
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install Net_Sieve
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install Auth_SASL
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install Net_DNS2
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/horde
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/Horde_Memcache
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/imp
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/turba
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/kronolith
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/mnemo
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/nag
/srv/www/vhosts/horde/pear/pear -c /srv/www/vhosts/horde/pear.conf install horde/ingo
Step#11 Initialize the configuration.
cp  /srv/www/vhosts/horde/config/conf.php.dist  /srv/www/vhosts/horde/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/config/conf.php
touch /srv/www/vhosts/horde/imp/config/conf.php
touch /srv/www/vhosts/horde/ingo/config/conf.php
touch /srv/www/vhosts/horde/turba/config/conf.php
touch /srv/www/vhosts/horde/kronolith/config/conf.php
touch /srv/www/vhosts/horde/nag/config/conf.php
touch /srv/www/vhosts/horde/mnemo/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/imp/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/ingo/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/turba/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/kronolith/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/nag/config/conf.php
setfacl -m u:apache:rw /srv/www/vhosts/horde/mnemo/config/conf.php
Step#12 Enable name based virtual hosting.
Edit the /etc/httpd/conf/httpd.conf file and uncomment the line reading "NameVirtualHost *:80".

Step#13  Create a virtual host entry for the Horde instance. If you have a server-name / domain-name you should substitute that for "horde.example.com". Otherwise if this instance is merely for testing/development adding horde.example.com to your workstation's /etc/hosts file should be sufficient to allow you to access the instance. The domain "example.com" will never be issued as an actual domain (see RFC2606) so it is safe to use for development deployments.  Depending on your site's policies you may want to configure custom logging for this virtual host.
(cat <<EOF
<virtualhost *:80>
    ServerAdmin webmaster@horde.example.com
    ServerName horde.example.com
    ServerAlias horde
    DocumentRoot /srv/www/vhosts/horde
    <directory /srv/www/vhosts/horde>
       Options Indexes Includes FollowSymLinks
       Order allow,deny
       Allow from all
    </directory>
   php_value include_path /srv/www/vhosts/horde/pear/php
   SetEnv PHP_PEAR_SYSCONF_DIR /srv/www/vhosts/horde
</virtualhost>
EOF
) > /etc/httpd/conf.d/x-vhost-horde.conf
Step#14 Start the web server (Apache) and Memcache daemon.
service httpd start
chkconfig httpd on
service memcached start
chkconfig memcached on
Step#15 You should now be able to hit the CentOS6 instance with your web-browser and automatically be logged in as the Horde administrator!  Go the the Administration / Configuration page via the left-hand menu and you should see a list of the installed Horde applications as well as the first level of Horde modules that provide services to those applications (such as "Horde_Alarm", "Horde_Activesync", etc...).  If you don't see those additional Horde modules listed then something went wrong with your PEAR installation; start over and carfully watch the output of the commands for errors or warnings.



Step#16 Generate new configurations for all applications; to perform this function click the "Update all configuration" button. This will fill in the various conf.php files we created in Step#11.

Step#17 Provision a PostgreSQL database for use by the Horde instance. Caution: If you are reusing a PostgreSQL instance from other applications do not perform the "service postgresql initdb" command.
yum install postgresql-server
service postgresql initdb
service postgresql start
sudo  -u postgres createuser --no-password --no-createdb --no-createrole --no-superuser horde4
sudo  -u postgres createdb -E utf-8 -O horde4 horde4
Once the database is provision you need to allow the Horde instance to connect to the database. For simplicity of this example we are connecting to the instance of PostgreSQL on the localhost so we will simply change the configuration to trust local connections. For production deployments at least a password should be configured for the connection. To grant access edit the /var/lib/pgsql/data/pg_hba.conf file and change "ident" to "trust" on the line reading "host    all         all         127.0.0.1/32".  Then restart the PostgreSQL database so that it rereads this file: "service postgresql restart"

Step#18 Configure the database connectivity of the Horde instance. Now that Horde is up and running subsequent configuration is simple. Select Administration / Configuration from the left-hand menu. From the list of applications select "Horde" and then choose the "Database" tab.
  • For database type choose "PostgreSQL"
  • Check the box enabling persistent connections.
  • For "username" enter "horde4"
  • Change protocol to "TCP/IP"
  • For "hostspec" and "port" enter "127.0.0.1" and "5432".
  • For "database" enter "horde4".
  • Leave "charset" as "utf-8" and "splitread" as "Disabled"
  • Once the form is filled in click the "Generate Horde Configuration"

Step#19: Now click the "Update all DB schemas" button; this will initialize the database with the required tables.  Every time applications are updated this button will allow the database schema to be automatically updated. This first time you initialize Horde you should perform this operation until no more database schema error or notices appear - typically this requires performing this operation twice.
Step#20The last steps involved in configuring the base Horde configuration is to enable a caching system to accelerate performance. For this example we are using the memcache service we enabled in Step#14. Navigate to Administration / Configuration, select the Horde application, and then choose the Memcache Server tab.
  • Change the status to "Enabled"
  • For "hostspec" and "port" enter "127.0.0.1" and "11211".  These are the default Memcache configuration parameters.
  • Enable persistent connections by checking the "persistent" box
  • Change to the "Cache System" tab.
  • For the cache system driver select "Use a Memcache server".
  • Click the "Generate Horde Configuration" button.
Your Horde configuration is now configured with database connectivity and an active caching system.  In subsequent articled the procedure for configuring specific applications and enabling OpenGroupware Coils integration.

by whitemice (noreply@blogger.com) at October 24, 2011 11:28 AM

October 21, 2011

Whitemice Consulting

Changing Terminal Services License Mode

You are provisioning a Window 2008R2 server for remote desktop service; you've configured the terminal services license manager in one mode [ device | user ].  But when you receive the license documentation you discover that the CALs purchased were for the other mode.  Then the Windows terminal license server manager tells you to change the mode of the license server.... but there is no obvious way to change the mode [because Windows is user-friendly!].  One option is to go old-school - hack the registry!.

First, stop the license server. Then in regedit change the value of the "LicensingMode" key in the "HKEY_LOCAL_MACHINE \ System \CurrentControlSet \ Control \ Terminal Server \ RCM \ LicensingCore" collection.  A value of "2" indicates per device licensing and a value of "4" indicates a value of per user licensing.  Then reboot.

by whitemice (noreply@blogger.com) at October 21, 2011 07:29 AM

October 13, 2011

Whitemice Consulting

Finding Address Coordinates using Python, SOAP, & the Bing Maps API

Bing maps provides a SOAP API that can be easily accessed via the Python suds module.  Using the API it is trivial to retrieve the coordinates of a postal address.  The only requirement is to acquire a Bing API application key; this process is free, quick, and simple.


import sys, urllib2, suds

if __name__ == '__main__': 
    url = 'http://dev.virtualearth.net/webservices/v1/geocodeservice/geocodeservice.svc?wsdl'
   
    client = suds.client.Client(url)
    client.set_options(port='BasicHttpBinding_IGeocodeService')
    request = client.factory.create('GeocodeRequest')

    credentials = client.factory.create('ns0:Credentials')
    credentials.ApplicationId = 'YOUR-APPLICATION-KEY'
    request.Credentials = credentials

    #Address
    address = client.factory.create('ns0:Address')
    address.AddressLine = "535 Shirley St. NE"
    address.AdminDistrict = "Michigan"
    address.Locality = "Grand Rapids"     
    address.CountryRegion = "United States"
    request.Address = address

    try:       
        response = client.service.Geocode(request)   
    except suds.client.WebFault, e:       
        print "ERROR!"       
        print(e)
        sys.exit(1)

    locations = response['Results']['GeocodeResult'][0]['Locations']['GeocodeLocation']
    for location in locations:       
        print(location)


If you need to make the request via an HTTP proxy server expand the line client = suds.client.Client(url) to:


    proxy = urllib2.ProxyHandler({'http': 'http://YOUR-PROXY-SERVER:3128'})\
    transport = suds.transport.http.HttpTransport()
    transport.urlopener = urllib2.build_opener(proxy)
    client = suds.client.Client(url, transport=transport)


The results will be Bing API GeocodeLocation objects that have an Longitude and Latitude properties.  Note that you may receive multiple coordinates for an address as there are multiple mechanism for locating an address; the method corresponding to the coordinates is a string in the CalculationMethod property of the GeocodeLocation objects.

by whitemice (noreply@blogger.com) at October 13, 2011 11:11 AM