Want to use Amazon S3 for your Spotfire Library? Read this first.

I’m not one for clickbait so my apologies to the esoteric group this pertains to. The quick and dirty of it is that you can’t use TIBCO’s ./config.sh tool for connecting the library to an external source like Amazon S3. Ordinarily the code is this:

$ ./config.sh config-library-external-s3-storage –bucket-name=MYBUCKET –access-key=MYACCESSKEY –secret-key=MYSECRET

This won’t work. Spotfire Server will try to connect and give you something like:

Connecting to the library…OK

Retrieving items from database…OK

Retrieving items from external storage…Unable to execute HTTP request: rai-exchangeai.s3-us-west

Is that a malformed URL for an Amazon S3 API call? You bet. Open up your configuration.xml and fancy your own URL by changing the region value to s3-us-west-2.amazonaws.com. Spotfire doesn’t include an example in the installation guide so this is my gift to you.

Spark lines can show the shape of highly variable data in a small amount of space

In Spotfire Graphical Table visualization, the use of sparklines is a fantastic way to quickly visualize our data in table format. But, what if we have highly variable data in which it would be better to use a logarithmic scale on the Y-axes? Note, there is no option for using a logarithmic scale on the spark line axes visualizations. We have two options here: to use multiple scales or write a custom expression with multiple scales.

One of the main uses of sparklines is to show the “shape” of our data. If our data range is less variable, then a single arithmetic scale for all sparkline axes is fine. However, in the case below we need to use a different arithmetic scale for each spark line in the column to honor the high variability of the data.

Go to Properties of Graphical Table > Axes and select spark line column as seen below. Now select Settings button for that spark line column.

Then, select Axes and change radio button under “Y-axis scale” from “One scale for all sparklines in this column” to “Multiple scales.” We do the same for all spark line columns with highly variable data we want to “compare,” as seen in the second spark line column in the Graphical Table below.

Note the huge improvement in being able to see the “shape” of our data with “Multiple scales” selected, compared to the first visualization above.

Next, if we want a Log or Logarithmic scale, we can easily write a custom expression as seen below by right mouse clicking of Y-axis name in Sparkline Settings.

Insert Log function Custom Expression, then hit Okay.

Compare final Log Scale Graphical Table below to previous two arithmetic Tables above. Note visualization below has “Multiple scales.”

Finally, if we use “One scale for all sparklines in this column” instead of “Multiple scales” the results may not show enough differentiation especially if you have extreme outliers. Compare this last log image with our first arithmetic one.

Addendum to Azure Install of Spotfire Server

Microsoft Azure doesn’t let you switch databases: “USE statement is not supported to switch between databases. Use a new connection to connect to a different database.” The USE command is used in the create_databases scripts in the beginning of the installation. This is a known bug for Azure, take note here: https://support.microsoft.com/en-us/kb/2292807.

Your best bet is to do the following:

  • Add -dspotfire_server to the populate and create_user sqlcmds
  • Comment out the USE commands and connect directly.

Good luck!

Making a QQ Plot in Spotfire with TERR

QQ Plots are a standard visualization that compares the distribution of your data under study to the normal distribution. Since most statistical tests assume normality, the QQ Plot is an important diagnostic visualization during any analysis of uni-variate or multi-variate studies. We had a previous post that made a QQ Plot using custom expressions, and in this post we will show how to do it in TERR.

Read More

4 Tips for GIS in Spotfire

In the oil and gas industry, ArcGIS is king. In terms of capabilities there’s no question that when you see a map lying around a corporate office, it was printed from ArcGIS. Over the years Spotfire has done quite a bit in the way of There’s quite a bit of Those of you handy with Spotfire may know the difficulties in replicating the large graphs. Below I’ve included some tips for those Spotfire developers that have found themselves crossing into that area.

When you get the link from them, be sure that it ends with /MapServer/WMSServer?request=GetCapabilities&service=WMS. This is key, otherwise you will be nosing around the MapServer with no success.

Understand WMS Layers

While Spotfire handles shapefiles, you may find youfself asking how can I create more dynamic maps without all these tables? WMS layers are the answer to that. If your ArcGIS team already has a MapServer, ask them to publish WMS services for the layers that you want. For example if you are asking for leases be sure to recommend the color and outline that you are looking for. WMS layers can be stacked on top of each other much like in ArcGIS, but as far as data goes, the power truly likes in the marker plotting in Spotfire.

Set the Zoom Visibility Controls

If you have multiple layers in the map chart, you will want to control whether some layers should be visible at certain zoom levels. For example, if you have feature layers that encompass larger portions of the United States, they may not be necessary at a well level. Use the zoom visibility feature to reduce the impact of these layers at a higher zoom:

Printing the Big Picture

This was a bit of a personal journey and by that I mean trial and error. My colleague recommended the simplest method by far, export the map chart as a PDF, noting to set the paper size to A0. For us Americans, I recommend the following infographic:

An A0 -sized landscape PDF export is just about what your typical land management executive wants to see for their particular areas. Export to PDF, print on the plotter, done.

Caching for Performance

Be sure to cache these layers as well, performance can be an issue when you are dynamically pulling more than one WMS layer. This also depends upon your latency as well.

That’s all for now! Let me know if you guys have any more advice on the topic!

Lucas