Part 3 – Improve Spotfire DXP Load Time

  • Do your DXPs take forever to load?
  • Would you like faster DXP load time?

Of course the answers to those questions are “Yes!”, and this is probably the post most of you have been waiting for.  This is the final part of a three part series on improving load times.  It will focus on how to improve DXP load time using cached information links and Scheduled Updates.  Last week’s post discussed how to improve Web Player load time using Scheduled Updates.  Use the link below to check out the previous weeks’ posts.  I also highly recommend checking out the comments sections.  Other folks have posted quality commentary that may be useful to your particular situation.

Part 1 — Troubleshooting Long Load Times

Part 2 — Improving Web Player Load Time with Scheduled Updates

The Prelude

So, how do you speed up DXP load time?  There are two parts to the process.

  1. Modify existing information links or create new information links that are cached.
  2. Create a Scheduled Update set to run early in the morning before your people get into the office.

The Scheduled Update will take a while to run (as long as it “used to” take).  However, once the Scheduled Update has completed, the information link is cached and all subsequent data pulls will be faster.  Just make sure to configure the Scheduled Update to run at an interval that makes sense with the cache period.  Now, let’s look at how to actually set it up.

 

Cached Information Links

You’ll find a section on Caching in the Information Designer, at the very bottom of all the options for an information link.

Mandatory

  • Check the box to make the information link cacheable.
  • Define the timeout period in seconds.

After the query executes, any subsequent queries pulled within this window of time will pull from cache, which will be faster.  After this period expires, the next query will ping the data source instead of returning from cache.

Optional

  • If your information link contains data from more than one data source, select which data source the validation query should be run against.

Improve DXP Load with Caching

Noteable

  • Spotfire Help says, “The information link should not be set as cacheable if the result of the information link is personalized or depending on things beyond the control of Information Services (for example, database row level security).

Scheduled Updates

Requirements & Prerequisites

  • Spotfire Server –> In order to run Scheduled Updates, you must select a DXP from the Spotfire Library, which means you need Spotfire Server.  Thus, if you are using the desktop product, this will unfortunately not be of much help.
  • Web Player –> Scheduled Updates are primarily for speeding up Web Player loading, and the DXP must load successfully on the Web Player to run a Scheduled Update.  Thus, check to make sure the DXP loads on Web Player before configuring the Scheduled Update.  If the DXP doesn’t load into Web Player, the Scheduled Update will not work.
  • Don’t recommend prompts on information links -> Scheduled Updates can work for DXPs with prompts, but (and this is a big BUT) it will open the DXP using the responses that were used when the DXP was last opened and saved.  Thus, if you have a process for opening and saving, it could work.  If you don’t, data loads could be messy and inconsistent.  Unfortunately, this is a deal breaker for many DXPs, as prompts are very common, especially with large data sets.

Configuration

Scheduled Updates are configured in the administration console, in the Scheduling & Routing section.  Configuring Scheduled Updates is a two step process.

  1. Build the schedule (Use Create Schedule button)
  2. Create the rule (Use Create Rule button)

Create the Schedule

  1. Give the schedule a name.  I like to indicate how often and/or when it runs.
  2. Specify which days of the week the job should run (only daily or weekly jobs are possible).
  3. Define the hours the job should run (ex. between 6am and 6pm).
  4. Select the time zone.
  5. Specify the reload interval (i.e. check for updates).

Improve DXP Load with Scheduled Updates

Create the Rule

  1. Specify the type of rule.  In this case, the rule type is FILE.
  2. Give the rule a name.  For example, “Early Morning 1hr Refresh” is a good name for something that kicks off early and refreshes every hour.
  3. Browse through the library to find the DXP.
  4. Select a resource pool and specify the number of instances.  These settings will vary depending on how your web player is setup.
  5. Set a priority.  For example, if you are running 10 scheduled updates, prioritize them 0-9 with 0 as the highest priority.
  6. Select which schedule should apply to the update.  Note, even though I showed you how to create a rule first, you can also create schedules in this screen.  I prefer to setup schedules in advance so I can see how a new schedule fits into what’s already being loaded.

Improve DXP Load with Scheduled Updates

 

Expected Results

Now, you may be asking — What kind of performance improvement can I expect to see?  Of course, the answer to this question will vary depending on a number of factors including the number of records in the information link and how many columns the information link has (Spotfire is optimized for tall and skinny tables).  I conducted two timed tests.

Test Rows Columns Initial Load Time Improved Load Time
Test 1 9,724,829 65 8:16 (min) 2 min approx*
Test 2 10,268,435 64 13:15 (min) 3 min approx*

*Less than precise note taking ><

I was more than happy with these results. For the second test, I took 10 minutes off my load time!

In Conclusion…

I want to note that there are a few other ways to improve load times not covered in this series.

  1. As indicated in the comments of these posts, you can use Automation Services. Unfortunately, Spotfire does not include Automation Services out of the box.  It must be purchased separately, although it is well integrated.
  2. Data can also be loaded “on demand.”  The idea behind loading data “on demand” is that you load a minimal amount of data at first, and then through marking or a property control, trigger a second query that pulls more detailed but also more specific data.  I’ll post on this soon.
  3. A direct database connection is going to be faster than an information link (generally speaking).  However, not all companies allow users to make direct database connections.

I hope you found this series informative and helpful!

One thought on “Part 3 – Improve Spotfire DXP Load Time

Leave a Comment

Your email address will not be published. Required fields are marked *