Syncing NIS and LDAP

Plans for migrating from NIS to LDAP vary widely. In some environments, the move can be fairly immediate: set up the client machines, sync the NIS maps to the LDAP Directory System Agent (DSA), shut down NIS, and cross your fingers. However, for a vast majority of cases, administrators want to have a little more of a contingency plan. They like to keep NIS and LDAP in sync, just in case a script turns out to be buggy, or they have client troubles forcing some machines to continue to rely on NIS until they find a solution. In this article I'll go over some tools I've found, and some code I've cobbled together, to make syncing NIS and LDAP a little easier.

For purposes of brevity, I'm going to cover syncing the NIS passwd and group maps with LDAP People and Groups trees. I'm using OpenLDAP and standard schemas for my LDAP objects, so nothing here should look weird if you're using the schema files supplied with the OpenLDAP DSA software.

Which way to go?

There are tools that can help you turn Lightweight Directory Interchange Format (LDIF, LDAP's standard data format) into NIS-compliant map data, and others to help turn NIS maps into LDIF. None of them will be a 100% solution to your needs. For example, PADL makes a set of Migration Tools so popular by now that they're included with some Linux distributions. I found them largely inapplicable to my environment, through no fault of the fine folks at PADL Software. In spite of the possible shortcomings when applied to your environment, it's a good idea to go over what's out there that might help you along.

The first thing you need to decide is which data source will be the canonical source of information. Without knowing this, you don't really have a ready response to the two data sources being out of sync! If LDAP and NIS disagree on a user attribute, which one is right, and which needs to be updated? The answer to this question will decide whether you'll be converting NIS data to LDAP data or vice versa. I'll show you a couple of tools that'll be useful in either case, and a couple that'll help you no matter which way you're going.

Generic is better

The best tools are those which are generic in nature and can be applied universally, no matter what you're doing. Two such tools are ldifsort.pl and ldifdiff.pl, listed along with the other Perl Net::LDAP tools at cpan.org.

These tools don't care about much of anything except that they're acting on an LDIF file. If you have two LDIF files from the same LDAP tree, you can use ldifsort.pl to sort it on whatever key you want, and then use ldifdiff.pl to output the changes necessary to make the out-of-date tree look like the up-to-date tree. You can immediately run ldapmodify, feeding it the output from ldifdiff.pl, and the changes will be made.

I use both of these tools in a scheme that takes NIS data and uses it to update LDAP. I cat out the NIS maps, run them through an awk script to change them to valid LDIF format, then grab the current LDAP content, sort both LDIF files, and do a diff -- et voila! I run ldapmodify on the output, and I'm up to date!

If you maintain a Java installation on the machine you run your admin scripts from, you might also be interested in DSML Tools, a collection of extremely useful extract, transfer, and load (ETL) tools specifically for LDAP data. These tools do everything that I use ldapdiff and ldapsort for, and in addition, the tools will apply the changes for me, without my having to run ldapmodify, which is very nice.

Evaluate these tools if a) you plan to keep NIS as the canonical source of data, since you'll update LDAP from NIS sources, or b) LDAP will be the canonical source of data, since you still have to do the initial population of LDAP from NIS sources. Even if you're going to update NIS from LDAP sources, these generic tools can help by creating a diff in a standard format which can be further parsed into an updated NIS map. I'd rather write only some of the code than all of it.

More specific tools

As noted, there are more specific tools you might find handy, either as a one-stop-shop solution, or as a foundation to a more customized solution. The most popular one is probably the aforementioned PADL tools. This is a suite of tools, really, designed to be generic in the sense that they use NIS or local file data stores to create generic LDAP objects. The nice thing about these tools is that they're all open source Perl scripts, so they have served me well as quick and dirty Swiss Army knives to get some work done. Just hack the scripts to output to a file, and hack in the objectclasses you use, and you'll probably come out with something useful. The downside is that there are quite a number of scripts, and they can take a while to get a good handle on.

Another specific tool is called ldap2nis, which aims to make the task of updating NIS from LDAP sources very easy. This is actually written, in C, by Adam Tauno Williams, who has written some excellent and useful documentation on integrating and administering OpenLDAP. If you're looking to get started on a migration, or just want to know more about OpenLDAP and what can be done with it, whether you're a home user or an enterprise administrator, I also urge you to read his LDAP PDF document.

In short, ldap2nis gets pointed at an LDAP DSA, and outputs a makedbm-compatible NIS map to standard out. It's a great tool, but it doesn't take a username and password, as far as I can tell, which means if you aren't using NIS shadow maps, and your LDAP password attributes are shielded from casual view, you won't have them in the NIS-ified output. Even if it took a username and password, it doesn't appear to support TLS or SSL. I'm actually working on hacking this into the code, which I'll dutifully send to Mr. Williams when I'm done. First, though, I need documentation on the OpenLDAP ldap_start_tls_s function. Anyone? ;-)

In closing

Of course, it's impossible to cover every tool here. I only touched on tools that are scriptable and usable from the command line. Certain functions may exist within GUI tools to help you import data, but I don't know of any that'll take that data from NIS/NIS+ or local files. If you maintain code that performs ETL-like functions going from NIS to LDAP or vice versa, you are encouraged to share them, as I will mine. I'll be posting my finished NIS-to-LDAP tool on my geek site when it's all finished.