Simon Fell > Its just code > January 2006

Monday, January 30, 2006

WS-FTP

WS-FTP, so many bad ideas in a single project, nice going !. Lets see, it sends chunks of a file over SOAP, it doesn't use any binary packaging scheme for this, so you take a 4/3 bandwidth / filesize hit (i.e. you spend at least 4Mb of bandwidth to send a 3Mb file). The chunking is done via multiple SOAP calls, so you're spending lots of round trips (remember, round trips are evil). All this to get a chunked upload, when in fact HTTP supports this out of the box, you can do this in a single HTTP call using PUT with chunked encoding (if you don't know the starting size). I guess you end up in this mess trying to paste over the holes in the .NET http client. No wonder Web Services have such a bad name.

You know its bad when I feel like I'm channeling Mark Baker!

Wednesday, January 04, 2006

Share and Enjoy

Sforce 7.0 aka Winter '06 goes live this weekend, the Whats new in Winter '06 outlines what's coming up, here's some of the things I think are interesting.

Improved Performance, requests hitting the 7.0 API are handled by a completely new SOAP stack, the new stack was written from the ground up purely to be Salesforces server side soap processor, its is extensible just enough to do its job and no more (many soap tools are extremely extensible, and it comes at a considerable cost), we concentrated hard on perf both response time and scalability. The open source soap stack that we used previously has particularly bad scalability issues, it uses a lot of memory to process a request, which leads to a high level of GC related load, and a reduced concurrency ability from transient out of memory exceptions. Reduced GC load is good for us, and improved response times are good for you, in some testing I did I saw response times (for the entire request not just the soap processing part) as much as 50% faster than the earlier stack. In addition creates / inserts and upsert calls for 7.0 by default will no longer update the application MRU, which can further increase write performance by upto 25%, this is a great win for back end integrations that don't run in the context of a end user of the application. (If you have an app that extends the Salesforce UI using the API, then you should send up the MruHeader in your request so that the MRU does get updated).

Upsert + External ID, there's so many uses for this its hard to know where to start. The External Id is a flag on custom fields, when you turn it on, we index that field (improved query performance for filtering on that field), in addition it now allows you to use the upsert call which uses that external Id as the key instead of the Salesforce Id. If you're syncing additional data sources with Salesforce, it makes that process much easier to handle. On additional use is to an external Id field as an operation identifier, SOAP over HTTP is not 100% reliable, if you make a create call and the response never makes it back to the client, its hard to work out if the data got created or not, by adding an operation Id custom field, you can make all your creates calls via the upsert call, simply assign a unique id (say a GUID) to the create you're trying to do and call upsert. If something goes wrong and the response is lost in transit, you can simply make the upsert call again, if the original insert never happened, then there won't be a row for that GUID and we'll do the insert, or if we did the original insert, the row will now be there, and we'll just update it. This can drastically simplify handling failure conditions.

Formula fields with results that are really HTML. You can do some cool things with Formula fields, including the HYPERLINK and IMAGE functions, but if you queried one of these from the API you'd get some weirdly formatted result that was neither HTML or plain text, and to make things worse, other than the formula flag in describe you didn't know if you were dealing with one of these weird fields or not. In 7.0 its all much cleaner, query calls for formulas that generate HTML will return the actual resulting html, and an additional flag in the describe results for that field will indicate that this particular field is returning html formatted values. This makes doing the right thing WRT to html encoding and handling scripting injection when re-presenting data via a web page much easier.

There's a ton of other new stuff, check out the tech note and start thinking about switching to 7.0

Now, if you're making that switch to 7.0 here's a few things that might trip you up, be cafeful!

• If you hand crafted your client soap stack, then there's a few things to watch for.
  • The stack requires that the SOAPAction HTTP header be present and not be completely blank. (based on the WSDL, you should be sending SOAPAction : "")
  • The EntityHistory sObject uses the xsd:anyType type for the old & new value fields, (xsd:anyType in the wsdl, anyType as the type in the describe results), this requires you to look for the xsi:type attribute in the actual soap message to determine the actual type.
  • The response serialization has some optimizations in it, first insignificant whitespace is no longer serialized (whitespace between elements), secondly all xml namespace declarations are declared just once in the root envelope element. Neither of these should affect you, unless you are doing something seriously dodgy like not parsing the response with an xml parser. In tests, these 2 changes in the response can cut the size of the response by upto 35%. (bytes over the wire are expensive, this is a good win for tools that don't do compression).
• the Delete call will no longer report success when you try and delete an id that doesn't exist (or has already been deleted), it instead returns a INVALID_CROSS_REFERENCE_KEY status code, in addition you can now only delete 200 items at a time (instead of 2000). Delete is a pretty expensive operation, the reduced batch size helps get you a completed response before timeouts start to kick in.
• The UI now supports dependent picklists, one picklists set of values can depend upon the value of another picklist. If you're presenting a UI based on the picklist values from the describe call, then you'll want to decode the controllerName and validFor fields in the describe so that you can show the same subset of picklist values as the application UI.
• The batchSize header in QueryOptions is now even more of a hint that it used to be, if you're using really small batch sizes be prepared for the query result to actually contain more records that the batch size (in previous versions we might make the batch size smaller than the batchSize header, but never larger, now we will).

All these changes and additions are purely for Sforce 7.0, if you have a previous API client written against an earlier Sforce version, it should continue to function as it does today.

Monday, January 02, 2006

Santa's Santos

I was good this year and Santa brought me a Bodum Santos, a vacuum pot coffee brewer, we set it up and tried it out yesterday, its a very cool contraption, its just fascinating to watch it push the water up into the top pot, mix with the grinds, brew the coffee, then have the brewed coffee drop back into the bottom pot, with all the grinds left in the top. It does this purely with a mechanical spring and vapor pressure, not a single electrical gizmo in sight, great to watch in action and quite the contrast to the high tech espresso machine I normally use. The resulting coffee had a very clean & crisp taste, vastly better than the more popular drip coffee results, and given the nature of it I was very surprised that there were practically no grinds left in the brewed coffee. If you normally drink drip and are looking for something that’s both better and a little different, check them out. Sweet Marias has a range of vac pots available, the Cona ones look realy nice. Mark Prince (the geek in coffeegeek.com) wrote a great vac pot faq if you want to know more.