Violating My Space - Client Integration

If you work in an environment where you have to integrate with external systems, such as clients or vendors, there's a significant chance you've felt the subtle encroachment. In terms of system architecture, this is a kind of violation (albeit necessary) of one's personal space. If you're anything like me, you're somewhat protective of the systems you work on. You care about making compensations for external entities.

Even as I write this, I can't help but think it's a coupling problem; they're too close. The considerations of external systems belongs in a highly quarantined detention facility, far from the core business logic of one's code. Let's call this detention facility the client integration end point.

Working with any number of external entities (to which are you beholden to as a service provider) means that you have to make compromises. In fact, it seems as if most of the compromises are always on our end. I mean, after all, the client is footing the bill and sometimes, no matter how hard you try, they're set on doing things their way. Let's take an example.

You have a system that deals with data. Your clients are going to send you data and you need to import it (whatever that means). You store data in UTF-8, internally. One client like ISO-8859-1, another likes ISO-2022-JP, and so on. Moreover, one client prefers SFTP file delivery, another is interested in making API calls and controlling things from their end. You, on the other hand, just put in your two weeks notice.

This isn't entirely out of the question, actually (surely most of you know that). You need to accommodate all of these different entities without loss of functionality in any of the given channels. You also know you could have N clients, where N is a really big number (because clearly, you're good at what you do). You need to work out efficiency issues and handle data loads of arbitrary size. It's no picnic.

I suppose the point of the example is that, even in such a contrived example, you need to handle:

• Varying transport mechanisms
• Configurable data recoding
• Different granularities of data updates (record level, data set level, etc.)

We didn't even talk about the format of files or APIs.

Sure, there are lots of standards around each one of these things, but I can tell you from experience everyone interprets them differently. Clients all use disparate platforms, libraries, and technologies that are out of their control. Sometimes, they, themselves, don't even know what their platform of choice is doing! Trust me. You can't tell them sorry, we just don't support that because there's money - a lot of it - at stake. They've already been sold on you and your technology and now it's time to put up (because shut up isn't an option anymore).

The worst part is that even if you're savey - you have your SOA enabled enterprise thing-a-joobers all wizzing and buzzing around, like they do - it might not make a damn bit of difference; you've still got the connector portion to define, which winds up being client-specific. No matter what, that's a lot of effort to be spent just so you can start doing business together.

There's only so much you can abstract away. Ultimately, if you receive format A and you want format B, you've got some work to do. The point is that it can be unpleasant. Find a good book, let it sink in, know what your clients want to do, and find what works for you.

I'm really interested in hearing from anyone who has significant client integration experience and learning more about the techniques other people use. Drop me a note.