Spotfire X New Feature: Filter as a Transformation (BEFORE Loading Data)

The other day, I attended a client’s Spotfire User meeting as Tibco representatives unveiled and highlighted new features in Spotfire X.  One specifically caught my eye:  the ability to filter as a transformation, before loading data.  This can be a huge time saver as the whole table will not need to load at start up.  You can only load what you need for that analysis.

Just that morning, I worked with a client, trying to filter data via the Data on Demand feature but it wasn’t behaving.  Frankly, I’ve always felt the feature a bit clunky.  But, now, there is no need to fight the Data on Demand interface or wrangle SQL filters in the Information Designer.  Of course, you can if you feel comfortable in those spaces; however, most users will love this very straightforward and intuitive method of filtering as a transformation.

You no longer need to ask a database expert to limit your view.  You can do it yourself.  To see how, read on…

First, go to the Data in analysis icon, pick your column, then set your filter.  This method was available via the Data Panel in previous Spotfire versions.  However, now you can right-click on the filter window and select Create Filter Transformation.

       

And, you’re done.

Seriously.

That’s it.

Next time you open the analysis, only the data you filtered to will load, reducing your load time to a fraction of the previous duration.

If you want to get creative with your filters, you can also use a custom expression to filter as a transformation.  To do so, go to the Data canvas (), select your table from the flow diagram, then click the plus symbol under the table name at the bottom of the page.  Then, choose Add transformations.

Once you see the window above, choose Filter rows in the Transformations dropdown, then Insert.

You will see the familiar custom expressions window in which you can type a limiting expression.

This expression serves just as the Limit data using expression feature in the Data tab of a visualization. Limit only to certain categories, remove certain categories, etc.

Except now the limiting applies to the data for the whole analysis and works as a transformation, before data is loaded.

Filter as transformation.  So easy.  So powerful.

5 thoughts on “Spotfire X New Feature: Filter as a Transformation (BEFORE Loading Data)

  1. Julie Sebby Reply

    We are upgrading today. Excited for new features but not the new interface.

  2. Chris Waters Reply

    Is that better that using the old “data on demand” feature that does what seems like exactly the same thing: limiting a data load from an information link by defining constraints when the data is first loaded? I am doing that sort of thing and have the constraints defined as properties. Seems to work well. How is this different?

    • Julie Sebby Reply

      This feature and data on demand aren’t quite the same thing. With data on demand, you only bring in what you need. People generally use on demand with large data sets because they are trying to improve the load time. With the filtering as a transformation, the data is brought in and then filtered out, so it doesn’t really help with load time.

  3. Jason May Reply

    The load time wouldn’t change initially, but every subsequent time after you create the filter, the load time would be reduced.

    Chris sometimes the Data on Demand can be rigid and a pain to work with. This filter feature gives users the ability to use the familiar filter interface and the data limiting expression capability gives a great deal of control. If On Demand works for you, stick with it though.

Leave a Reply

Your email address will not be published. Required fields are marked *