PuTTY “Access denied” before login to RHEL

January 10, 2012

Problem:

When you use PuTTY to SSH to a RHEL machine, you get an “Access denied” error displayed before the prompt for your password.

Solution:

Edit /etc/ssh/sshd_config and set GSSAPIAuthentication no

Then /etc/init.d/sshd restart

Filed under: Linux | Comments (0)

Locate a file, then cd to the location of that file

December 14, 2011

Sometimes the hardest part of improving your productivity is being able to notice that you’re doing something sub-optimal multiple times each day.

I was sitting hacking on some documents this morning when I realized that I frequently follow this usage pattern:

1. Locate a file, using the locate command—often a Linux configuration file of some sort.
2. Change directory to the directory containing that file, which is often a long way from the root directory or my current directory.
3. Do some stuff using the file.
4. Go back to what I was doing before.

A typical interaction:

w510:~/WIP 740$ locate s-pre-01.tex
/usr/local/context/tex/texmf-context/tex/context/base/s-pre-01.tex
w510:~/WIP 741$ cd /usr/local/context/tex/texmf-context/tex/context/base
w510:/usr/local/context/tex/texmf-context/tex/context/base 742$ cp s-pre-01.tex ~/WIP/
w510:/usr/local/context/tex/texmf-context/tex/context/base 743$ cd ~/WIP/
w510:~/WIP 744$

The problem here is that long path. I either have to type it with assistance from tab completion, or copy and paste it using the mouse.

I realized it would be really handy if I could do something like cd `locate s-pre-01.tex`

Of course, that doesn’t work for a couple of reasons, most notably that locate outputs a path to a file, not to a directory.

I checked to see if locate had an option to output only the directory name of the match, or if cd had an option to accept a filename and move to the same directory as the file. No on both counts.

Next, I checked to see if something like cdargs would solve the problem, but it seemed not.

I had the feeling a lot of other people had probably wanted to do what I wanted to do, so my next stop was Google. That turned up unhelpful monstrosities like cd "$(dirname "$(find / -type f -name ls | head -1)")"

Due diligence done, it looked like I wasn’t about to reinvent the wheel. I hacked together a couple of bash shell functions. My new improved interaction:

w510:~/WIP 757$ cdlocate s-pre-01.tex
w510:/usr/local/context/tex/texmf-context/tex/context/base 758$ cp s-pre-01.tex ~/WIP/
w510:/usr/local/context/tex/texmf-context/tex/context/base 759$ cd -
/home/meta/WIP
w510:~/WIP 760$

Note the use of cd -, an undocumented feature of bash which returns to the previous working directory. (It’s undocumented in help cd, at any rate.)

Filed under: Linux, System administration | Comments (0)

Random cool hack: Make your Ruby scripts work as man pages

October 27, 2011

Via careful use of nroff directives and comments, it’s possible to make a Ruby script that works both as a command line utility you can run, and as a man page. It’s a floor wax and a dessert topping.

This is handy, as it means you can distribute both the utility and the documentation in a single file, and symlink the script into the appropriate man page.

You can also, of course, look for the standard --help command line option, and make your utility call man with its own source file as argument to provide the help.

Note that if your utility makes extensive use of command line options, you’ll probably want to use GetoptLong.

Filed under: Ruby | Comments (0)

Notes 8.5.3: ⊕/⊖

October 25, 2011

Things that make me happy about Notes 8.5.3:

1. I can export my database designs to plain files, and put them in Git for version control.
2. I can grep all my source code with ack.
3. The built-in editor will let me see all the code for a design object in one window.
4. Hover-over display of documentation comments for functions.
5. Eclipse-style error summary.
6. Real HTML and XML generation capabilities.
7. HTML, ECMAScript and CSS in the Notes client.
8. Server and client side JavaScript. I look forward to the day when I never have to write any LotusScript ever again.

Things that make me unhappy about Notes 8.5.3:

1. Still no Linux design client.
2. Still no Mac design client.
3. No way to display bookmarks as icons in a grid. (And no, the old legacy Workspace is not an adequate substitute.)
4. I can’t believe they rewrote it from the ground up and didn’t make it fully multithreaded.
5. Errors are often reported on the wrong line number. That is, server-side error messages give a line number that may be 3 or 4 lines off from the actual place where the error occurred.
6. LotusScript code editor no longer autocompletes the end of a block when you open a block, so now I need to remember which keywords are used to end each different kind of block.
7. LotusScript code editor doesn’t clean up indentation automatically.

Filed under: Domino | Comments (0)

Notes/Domino 8.5.3: Using version control

October 21, 2011

One of the big new features in IBM Lotus Notes 8.5.3—specifically, in the Domino Designer—is support for external version control tools. To me, this is the single best thing ever to happen to Notes and Domino. Here’s how to use it.

Start off by opening your database in Domino Designer. Right-click the icon, choose Team Development from the popup, then choose Set Up Source Control for this Application.

Go through the wizard which appears. Give your project a name, and set the location on disk where files should be placed. (If you override the default, the location you choose should be a directory where you want the files to go, so you’ll probably want to create one.)

Now the magic happens. Notes writes out the entire design of the database, as a set of files neatly arranged in the following subdirectories under the project directory:

WebContent
Forms
Pages
Resources
Resources/Files
Resources/StyleSheets
Resources/Images
SharedElements
SharedElements/Columns
SharedElements/Subforms
SharedElements/Fields
SharedElements/Outlines
Views
Code
Code/ScriptLibraries
Code/Actions
Code/Agents
Code/WebServices
AppProperties
Framesets

Now you can open up the project directory that contains those folders, and stick the whole thing in version control. For example:

$ cd ~/workspace/MyProject
$ git init
$ git add *
$ git commit -m 'Initial import'

Congratulations, your Domino database is now under version control. You can tag releases, examine changes, fork and merge, and do all the other great stuff version control allows. I run RedMine at work, and push each bug fix as a commit to a repository on the machine. I put the number of the ticket in the commit message, and RedMine automatically cross references everything and makes the changes browsable.

Browsable? Yup, because most of the files Domino generates are XML, in Domino XML (DXL) format. There are still a few binary blobs, but I hear that these will gradually become DXL too. So changes to views, form fields, agents and the like can sensibly be examined in version control. And of course, all your LotusScript, ECMAScript, CSS and HTML resources are there in plain text.

This also means you can grep all your source code. Or even better, use ack, which by default will skip the GIFs, JPEGs and other binaries.

Behind the scenes, the Domino Designer watches the directories and the NSF file, and synchronizes changes—bidirectionally. Change something in the Design client, and it gets quietly updated on disk. You can edit a style sheet file in $PROJECT/Resources/StyleSheets using Vim; when you save it, Notes notices and updates the NSF. There’s a manual sync option for when you want to be absolutely certain everything’s in sync (say, before a commit), but for general development everything seems to work automagically.

This is the best thing to happen in the world of Notes development since… since ever. All I want now is a non-Windows Domino Designer and I’ll be happy.

Filed under: Domino | Comments (0)

Notes/Domino 8.5.3: Improving local replica efficiency

October 21, 2011

Recent versions of IBM Lotus Domino have included updates to the On-Disk Structure (ODS), the file format of Notes and Domino databases. However, if you have local replicas, they will not automatically be upgraded to the newer formats. In fact, even newly-created local replicas aren’t created in the new formats by default.

To fix this, add the following two lines to NOTES.INI while Notes isn’t running:

NSF_UpgradeODS=1
CREATE_R85_DATABASES=1

Now you can use the Admin client to compact your local replicas using copy mode, and it will upgrade them to 8.5 on-disk format, which is more space efficient and provides better performance.

I’ve no idea why this isn’t the default behavior. It made a huge difference to how snappy e-mail was for me.

Filed under: Domino | Comments (0)

NotesViewEntryCollection vs NotesViewNavigator

September 19, 2011

A common code pattern in IBM Lotus Notes / Domino applications is:

• Open a lookup view.
• Find all the rows with a particular key value in the first column.
• Loop through the rows pulling out data from the other columns.

It is the fastest way to look up data from multiple documents. It’s the LotusScript equivalent of using DBLookup to get the set of allowed values for a keyword field. The code typically looks something like this (deliberately trivial) example:


Sub Initialize
  Dim key As String
  key = "XHWG-7CNBJ4"
  Dim session As New NotesSession
  Dim db As NotesDatabase
  Set db = session.CurrentDatabase
  Dim view As NotesView
  Set view = db.GetView("RoomsByLocation")
  Dim vec As NotesViewEntryCollection
  Set vec = view.GetAllEntriesByKey(key)
  Dim ve As NotesViewEntry
  Set ve = vec.GetFirstEntry
  Dim count As Integer
  Dim roomnames As String
  While Not(ve Is Nothing)
    ' This is the start of the actual 'work' we want to do
    count = count + 1
    If roomnames <> "" Then
      roomnames = roomnames + ", "
    End If
    roomnames = roomnames + ve.ColumnValues(1)
    ' End of work, back to looping
    Set ve = vec.GetNextEntry(ve)
  Wend
  Print "Counted" + Str(count) + " using NotesViewEntryCollection: " + roomnames
End Sub


This code is (comparatively) fast because we don’t touch any NotesDocument objects, so the server doesn’t have to fetch the actual document data. Everything is done using the cached data in the view.

(As an aside, setting view.AutoUpdate = False makes the code significantly faster, if you can live with ignoring updates to data that occur while the code is running.)

However, recent changes in the Domino server mean that NotesViewEntryCollection is no longer the fastest way to access NotesViewEntry objects. The preferred API going forward is NotesViewNavigator, which has seen significant performance optimizations.

Thus today I found myself asking: how do I rewrite code that uses NotesViewEntryCollection, to make it use NotesViewNavigator instead?

My first problem was that NotesView doesn’t have an equivalent of GetAllEntriesByKey that returns a NotesViewNavigator object. There’s view.CreateViewNavFromCategory, but that only works on categories–not on sorted columns. This is actually the reverse of the situation with NotesViewEntryCollection, which doesn’t work reliably with categorized columns. In some cases I can make the appropriate view column categorized, but not always. So I needed a more general approach.

The answer seemed to be to find the first entry that matched the key, then create a view navigator from that point, then loop through checking that the first column continued to match the key. My first attempt looked something like this:


  Dim key As String
  key = "XHWG-7CNBJ4"
  Dim session As New NotesSession
  Dim db As NotesDatabase
  Set db = session.CurrentDatabase
  Dim view As NotesView
  Set view = db.GetView("RoomsByLocation")
  Dim ve As NotesViewEntry
  Set ve = view.GetEntryByKey(key)
  Dim count As Integer
  Dim roomnames As String
  If Not(ve Is Nothing) Then
    Dim vnav As NotesViewNavigator
    Set vnav = view.CreateViewNavFrom(ve)
    While Not(ve Is Nothing)
      If ve.ColumnValues(0) <> key Then
        Goto breakloop
      End If
      ' This is the start of the actual 'work' we want to do
      count = count + 1
      If roomnames <> "" Then
        roomnames = roomnames + ", "
      End If
      roomnames = roomnames + ve.ColumnValues(1)
      ' End of work, back to looping
      Set ve = vnav.GetNext(ve)
    Wend
  End If
breakloop:
  Print "Counted" + Str(count) + " using NotesViewEntryCollection: " + roomnames


A few things to note here. First of all, there’s no point in creating the NotesViewNavigator unless you get at least one match, so it goes inside the outermost If statement. Secondly, you can’t make checking the ColumnValues against the key a part of the While loop like this:


    While Not(ve Is Nothing) And ve.ColumnValues(0) = key


If you do that, you’ll run up against the feature that LotusScript always fully evaluates boolean expressions. That is, with the above line, ve.ColumnValues(0) will get tested against the value of key even if ve is Nothing. Hence you’ll get an error.

There were two things I didn’t like about my new code. The first was that it requires two levels of indentation, where previously one was sufficient. Call me picky, but I feel that a conceptually simple single looping operation should only need one level of indentation. The second problem, obviously, was that there was a Goto statement stinking the place up.

A little more work gave me this refactored version:


  Dim key As String
  key = "XHWG-7CNBJ4"
  Dim session As New NotesSession
  Dim db As NotesDatabase
  Set db = session.CurrentDatabase
  Dim view As NotesView
  Set view = db.GetView("RoomsByLocation")
  Dim vnav As NotesViewNavigator
  Dim ve As NotesViewEntry
  Set ve = view.GetEntryByKey(key)
  Dim count As Integer
  Dim roomnames As String
  Do While Not(ve Is Nothing)
    If ve.ColumnValues(0) <> key Then
      Exit Do
    End If
    If vnav Is Nothing Then
      Set vnav = view.CreateViewNavFrom(ve)
    End If
    ' This is the start of the actual 'work' we want to do
    count = count + 1
    If roomnames <> "" Then
      roomnames = roomnames + ", "
    End If
    roomnames = roomnames + ve.ColumnValues(1)
    ' End of work, back to looping
    Set ve = vnav.GetNext(ve)
  Loop
  Print "Counted" + Str(count) + " using NotesViewNavigator: " + roomnames


This code keeps the familiar “single while loop” structure, at the cost of 3 extra lines of code. It’s 6 lines longer than the NotesViewEntry version, but those lines are all at the start of the loop, which should make refactoring less error-prone.

So how good is the speedup? On small databases and in the Notes client, practically undetectable so far, but I’m looking forward to trying it on an 8GB database.

Even without performance gains, though, the revised code has the fringe benefit of added flexibility. You can match multiple keys, match partial keys without needing a different view, and so on.

Filed under: Domino, LotusScript | Comments (0)

SQL0805N Package “NULLID.SQLC2H21 0X4141414141504161″ was not found.

September 19, 2011

A quick FYI: IBM DB2 9.7 isn’t compatible with DB2 9.5 or 9.1 servers by default. The symptoms of the incompatibility are that some (but not all) queries result in an error which looks like:

SQL0805N Package “NULLID.SQLC2H21 0X4141414141504161″ was not found.

The solution is that the servers must be made compatible with the newer client by adding some binding permissions.

This incompatibility seems to extend to the pure Java Type IV driver as well. The problem in that case seems to be that the driver just returns no data; either that, or Eclipse is failing to report the error — I haven’t investigated too hard, I just downgraded to an older Type IV driver and everything worked.

Filed under: Java | Comments (0)

Sending Domino logs to syslog

July 19, 2011

As a Unix guy, the IBM Lotus Domino server log has always annoyed me. Since Domino is cross-platform, and Windows doesn’t have a syslog, Domino keeps its logs in a Domino database. While that makes some sense, it leads to colossal overheads; the log.nsf database is often 95% unused space. By switching my 90 days of logs to compressed syslog files, I was able to reclaim gigabytes of space.

In addition, because syslog files are just text, you can grep them, and use all kinds of open source tools to analyze them. You can use rsyslog to concentrate logs from multiple servers into a single set of logs. You can set up tasks to page you about problems. And so on.

So, I started with Daniel Nashed’s excellent rc_domino scripts, and crudely hacked them to make them pump the Domino console logs into a custom Ruby program which parses them and sends to them to syslog. You can get the code from GitHub via those links; I’ve been running it for several months now without any crashes or noticeable bugs.

Note that you still end up with events in log.nsf with this solution, but you can set it to a much shorter retention time. Hopefully at some point rc_domino will get cleaner support for console log processing. Ideally Domino would get official syslog support, but I won’t be holding my breath for that.

Filed under: Domino, Linux, Ruby, System administration | Comments (0)

It’s possible to be too helpful

July 1, 2011

I think this may be the best JavaScript error I’ve ever been given:

Uncaught TypeError: Object [object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object], has no method 'delete'

Thanks, Chrome. The variable name would have sufficed.

Filed under: JavaScript | Comments (0)