Data Without a Source

Typically a Qlik application will load its data from a data source, such as a database, API or flat file. Sometimes you just want to create something on the fly. The AUTOGENERATE function is a great way of doing this, and many of the applications on my Qlik Examples page use just this functionality. The function gives a specified number of rows of data that you can then work from. To create a table with a single field with values from 1 to 1,000:

LOAD
    RowNo() as Value
AUTOGENERATE(1000);

This may be useful in and of itself, but it is more likely you will need to do more. Read on to find out some of the things you can do.

Well, That’s A Bit Random

Often you will want to create some random values, and for this the rand() function is what you require. This function will return a number which falls somewhere between 0 and 1. Whilst at first this sounds a bit limited, with some very basic maths you can create many kinds of values.

So, to get a random integer between 0 and 999 you could do the following:

LOAD
    floor(rand()*1000) as Value
AUTOGENERATE(1000);

If you wanted to get a relatively random number of dimension values from a list you can multiply the random value by the number of values you have, round up to the nearest integer and then use a pick() function to choose which dimension value to pick:

LOAD
    pick(ceil(rand()*5),'Ben','Ian','Tom','Paul','Olly') as Person
AUTOGENERATE(1000);

Note with this approach though you will get roughly the same number of rows for each of the values, due to how random numbers work. If you want to have less regular number of rows you will need to introduce some bias.

If you want to generate random codes that look like they could be real (but are not) you can use the fact that the chr() function picks an ascii character based on a number. To get a code that looks like ABC123 you would have:

LOAD
    chr(65+floor(rand()*26))
        & chr(65+floor(rand()*26))
        & chr(65+floor(rand()*26))
        & num(ceil(rand()*999), '000') as PersonID
AUTOGENERATE(1000);

Given the number of possible values that this could generate you would most likely get all unique values. If you would prefer to guarantee that there are no duplicates you could use rowno() instead of rand(). If you wanted to cause more duplicates you could reduce the number of characters and the multipliers on each of the random values, perhaps reducing the letters to A to D and the digits to all be multiples of 9:

LOAD
    chr(65+floor(rand()*4))
        & chr(65+floor(rand()*4))
        & num(ceil(rand()*99)*9, '000') as DupedID
AUTOGENERATE(1000);

If you want to have data shown on a line chart you can generate dates, using the fact that dates are just numbers under the bonnet and we can therefore do maths on them. Again, if you want one row per date then use the rowno() function, or if you want more randomness around the rows per day use rand():

LOAD
    date(today()-(rowno()-1), 'DD MMM YYYY') as Date,
    date(today()-floor(rand()*90), 'DD MMM YYYY') as [Date Rolling 3m]
AUTOGENERATE(1000);

Creating dates in this way has been particularly useful in some of the demo apps I’ve created, as a simple reload will make the demo look fresh and current as all dates are in a period leading up to the current date.

That’s just a few examples of generating test data using AUTOGENERATE, but I am sure you can think of many more.

But Does It All Correlate?

I mentioned at the top that I had a recent requirement to test the correl() function, and generated some data to do this. For those who have note come across the statement before, it returns the aggregated correlation coefficient for two data sets. Or put into English, a value is returned which is close to 1 if there is a very strong positive correlation, close to -1 if there is a strong negative correlation and close to zero if there is very little correlation between two sets of values.

The function is called like this:

correl(aggr(sum(X), Dim), aggr(sum(Y1), Dim))

This is where Dim is a dimension value in the data and the two aggr statements create two datasets with a row for each dimension value. The sum(X) and sum(Y) expressions would usually be more complex calculations, but in this case it is simply the numbers generated in the load script.

The load script to make this expression work is as follows:

LOAD
    'Row ' & num(RowNo(), '#,##0') as Dim,
    RowNo() as X,
    RowNo() as Y1
AUTOGENERATE(1000);

Because we are using the row number for both the X and Y1 values there is an absolute positive correlation and the function returns 1, as it should.

For all subsequent tests we keep the Dim dimension and the X value, which always gives us a point along the bottom of the chart. For a negative correlation the value of Y can be calculated as:

1000 - RowNo() as Y2

So now when X is 0 Y is 1000 and vice versa. This is an absolute negative correlation, and the correl function returns -1, again as it should.

The rest of the values for Y which I tested for where as follows:

Rand() * 1000 as Y3

A totally random value, which gives a correlation coefficient that is very close to zero and a plot that is full of randomly placed dots.

RowNo() * Rand() as Y4

A random value between 0 and 1 multiplied by the X value, so that the Y value is random, but never greater than Y. This gives a correlation coefficient that is around 0.5 and a chart where the bottom right half is filled. This could be inverted, but I did not try that value.

(RowNo() * (Rand()-0.5)) + 500

A value that deviates from the centre of the chart on the Y axis by a random number that is plus or minus half of the value of Y. As the values fall above and below the line equally the correlation is very close to zero.

RowNo() + ((Rand()-0.5)*(RowNo()/1)) as Y6

This value is similar to the X=Y chart we started with, but with a random deviation from the line in either direction, up to half of the Y value. The divide by 1 is left in there as I was experimenting with having the deviation greater or lesser by dividing the row number. Correlation coefficient is approaching 1, but is not there.

500 + ((rand()*10)-5) as Y7

This value has a similar Y value for every value of X. I started off having it exactly the same for each X value, but the correl function requires some variance in the number or the spread will be zero causing a divide by zero and no correlation coefficient number. A random deviation of up to 5 in either direction provides this difference, but that deviation is small enough that the correlation coefficient is very close to zero.

500 + ((rand()*10)-5) as X2

The final expression is a bit different to all the others, as I wanted values from 1 to 1000 for all Y values in order (I use Y1 from above for this) and then have a calculation for the X value that is close to the same value, but off by a small margin. You will note the expression for X2 is identical to Y7, and I could have used the same field in the correl function, but it felt wrong to have two values labeled Y in the chart.

The resultant scatter charts and correlation coefficients for all of the derived values above can be seen in this screenshot (click to expand).

The entire load script for the Correl application looks like this:

LOAD
    'Row ' & num(RowNo(), '#,##0') as Dim,
    RowNo() as X,
    RowNo() as Y1,
    1000 - RowNo() as Y2,
    Rand() * 1000 as Y3,
    RowNo() * Rand() as Y4,
    (RowNo() * (Rand()-0.5)) + 500 as Y5,
    RowNo() + ((Rand()-0.5)*(RowNo()/1)) as Y6,
    500 + ((rand()*10)-5) as Y7,
    500 + ((rand()*10)-5) as X2
AUTOGENERATE(1000);

And the entire application can be downloaded from Qlik Community here:
https://community.qlik.com/t5/Qlik-Sense-Documents/Qlik-Sense-App-Create-Test-Data-For-Correl-Function-Using/ta-p/2139164

Hopefully that has given you some inspiration what you can do when you need to generate some test data. I would be very interested to hear of other thoughts you might have on the topic, so please do use the comments box below to let me know.

The post Generating Test Data In Qlik Load Scripts appeared first on .

Overview

For those that don’t know, if you have a QlikView Server implementation you can use Full User CALs on the server to licence a copy of QlikView Desktop in order to build applications. This means that a user can get twice the use of their User CAL. If you have a Full User CAL on the Server and also have a Desktop Licence for the same user you are essentially paying twice, and should review your licencing. Desktop Licences are still required if you don’t have a server, or are using QlikView NPrinting server (as the lease doesn’t always complete in time upon report creation).

Licences are leased from the QlikView Server when a document is opened in QlikView Desktop from that Server, a Full User CAL is used and Allow Licence Lease is enabled.

This post, lifted from our support portal, describes the process.

How To Lease a Licence From QlikView Server to a copy of QlikView Desktop

When you first download and install QlikView Desktop it will open up in Personal Edition. This is a full install of QlikView Desktop, but it is limited by its licence. In order to have full functionality on the desktop you must either apply a Desktop licence or lease a licence from a server.

If you have a Full CAL on a QlikView Server (via Access Point, for instance) then you most likely want to lease a licence.

A leased licence is managed by the server and can be used on the desktop for up to thirty days without being connected to the server. The lease is automatically renewed each time the Desktop client is opened up when on the same network as the server.

In order to lease a licence;

• Open QlikView from Start menu or TaskBar
• Select Open In Server from the start page
• This is in the bottom right in QV 11.2 SR6+
• It appears on the left hand side before SR6
• If the start page is not appearing select Show Start Page from the Help menu
• Enter address of your Access Point without the https:// in the Address box
• Click Open >>
• Browse apps on the server and open one you have access to
• You can now close this document

Whilst you are in Personal Edition this shows in the caption bar at the top of your QlikView Desktop app. This text is removed when a licence is successfully leased.

In order to lease a licence you must have a Full CAL on the Server and access to at least one app hosted on the server, and Allow Licence Lease must be ticked (in QMC under System \ Licences \ CALs \ General).

What if this doesn’t work

If you have followed the steps above and you are still having problems then try these troubleshooting tips from Qlik support:

https://community.qlik.com/t5/Official-Support-Articles/Troubleshooting-QlikView-License-Lease-Issues/ta-p/1710188

If you have any further tips, please add them in the comments at the foot of this post.

The post Back To Basics – QlikView Licence Lease appeared first on .

What Causes Orphaned Records?

Applications that show in upper case at the bottom of the list of apps on the Documents tab in QlikView Server are those that have been deleted, but still have hooks back into the QMC. Typically this is either a reload task or some Document CALs. If it’s the former then this task will run and fail on it’s old schedule – sending annoying failure mails. If it’s the latter then there are one or more CALs which can not be used on this app, or be re-assigned to others, wasting money.

How Do I Remove Them?

In order to remove the orphaned records you simply need to remove the CALs or reload schedule. Unfortunately this is not always as easy as it sounds. It’s good practice to always remove reload tasks and document CALs before you delete or rename an app. However, sometimes this doesn’t happen.

Without the documents being in the correct location it is not possible to get to the Reload tab or the Document CALs tab, this prevents the removal of the CALs or tasks after deletion. If you have just deleted the file then you can simply restore it, then remove anything associated with it and then delete it again. This will put you back where you want to be.

If you can’t just restore the file, but you know where it used to exist, then you can create a new QlikView app with exactly the same name – this will put the app back in the correct place and allow you to reach the configuration pages. There doesn’t need to be anything in the file, you can simply right click in the folder and select New / QlikView Document and create the file. You will need to open it in QlikView and save it back again before it appears in the QMC – without opening and saving it is simply a zero byte file that the QMC doesn’t recognise.

How Do I Create A File If I Don’t Know Where It Used To Be?

If the prior location of the file has been lost then you will need to find out where it used to be on the server. If you have log files from when the file was previously reloaded or accessed through Access Point then these should point you to the location you need to recreate the file. A bit more detective work is required if not.

Fortunately there is an excellent tool, called the QlikView Server CAL Manager, by David Vasseur that can help. Full details of this product can be found in the Qlik Community thread here:

[Thread Removed].

Among the various options in this tool (including allowing the clearing down of un-used Document CALs) is the ability to create an XML file of all documents hosted in the QMC – including orphans with CALs attached. This could direct you to knowing where to recreate the files.

Things get a bit trickier if it is the Mounted Folder has been removed, but if you can find what the folder was called (either from Access Point Logs or the CAL Manager) then that mount can be recreated and the missing documents put within it.

Conclusion

Hopefully if you have been blighted with orphaned QlikView apps the advice above will allow you to remove them. If not please feel free to post specifics in the comments below and someone (myself included) may be able to assist.

Thanks to Bill Markham who posed the question about removing orphans to me, and prompted the writing of this piece.

The post How To Remove Orphaned QlikView Apps appeared first on .

Don’t Remove Dimensions, Over Type

I’m sure you’ve been in the situation where you have set your Pivot Chart up beautifully. The alignment is right, pivot dimensions are correct, format expressions on the text – all perfect. You then realise that you made one slight slip when first setting up the chart, and you picked the wrong one of two similar dimensions. No problems, you think, just remove the incorrect dimension and then add in the right one. A couple of double clicks later and all is fixed. Except…

What you will find is that all the effort that has gone in to alignment and setting of other properties has gone. Worst case scenario the dimensions are now in a different order and all your pivoting needs to be re-done. Five minutes later you are back where you were before needing to make the switch.

There is however a better way.

If you click the Edit button that sits beneath the list of Dimensions you can simply type in the new dimension.

This button is generally used for editing Calculated Dimensions, but simply typing a dimension name is also allowed. You do need to be careful not to turn this into a calculated dimension with a single field (although it will work, my assumption would be performance could be hit a bit). Type in the field name, resisting the temptation to start with an equals sign or to add square brackets on fields with spaces. The latter feels a bit alien – and breaks both auto-complete and syntax completion – but it is the best way.

When the dimension is changed like this all settings relating to that dimension, such as layout properties, are maintained. This technique works the same for any chart type. Unfortunately it will not help you with Table Boxes. I will pretty much always use this for swapping dimensions as it also has the advantage that you don’t have to pick from the list on the left.

Copy and Paste Expressions

When thinking about the way that over-typing dimension names, and how it retains formatting, it reminded me of another neat feature that not many people seem to be aware of.

If you right click on an expression name a small context menu appears. On this menu you will see a Copy option.

Now, obviously you could click into the Expression Definition, do a Ctrl+A to copy the expression text, create a new expression and then paste in the code – this makes sense as you inevitably need to tweak the code. However, when you Copy and then right click in the space under the Expression list and click Paste something really neat happens. Not only do you get a copy of the expression code, the new column that is created has exactly the same properties as the source expression. This includes column width, alignment, number format etc.. This saves you having to set the formats and alignment on each column and it is also the simplest way of ensuring equal column widths – as there is no way to highlight multiple columns and drag them to the same size, as in Excel.

Conclusion

I hope that these two techniques are as useful to you as they have proven to be to clients that I have demonstrated them to.

If you have any quick tips of your own that you would like to share please feel free to do so in the comments box below.

The post Two Tips to Improve Your Qlik Life appeared first on .

Introduction

This series of posts looks at some of the features rarely blogged about as they are so second nature to many experienced QlikView bloggers. The sort of features I will use on a daily basis without thinking about it. These techniques are useful to revisit though and for anyone who has not come across these features before they are the things you really should become aware of. Preceding Loads deserve to be added to the list of features covered here.

What Is A Preceding Load?

As the name suggests, the preceding load happens in front of another load. You have probably used one already – even if you were not aware of it. When loading from an ODBC or OLEDB data source you have a SQL SELECT statement and the wizard will (optionally) add a LOAD section ahead of the SELECT. This is a basic preceding load. I would recommend that you always use one of these ahead of a database load – as it opens up a whole range of syntax that is not available in the SQL statement. So, if that is a simple preceding load, what can you do with a more complex one?

Potential Confusion

Before we get stuck into preceding loads fully it is important to understand the order things happen in your load script. In the main, script is executed from top to bottom, then let to right along the tabs (the tabs have no functional relevance are only there to tidy code). Loops and subroutines will alter this execution path, by design, but apart from that this statement holds true. The gotcha however is with LOAD blocks, which can be stacked together and execute from the bottom up. Picture the simple SQL preceding load:

The SQL part is executed and returned first and is then parsed by the LOAD statement above. This idea of loading from the bottom up holds true as we get into multiple layers of preceding loads, and it is important to keep in mind as we continue.

Why Use A Preceding Load?

Put simply, a preceding load allows you to use values derived in one part of the load in the one above it. To give a simple example, let’s say we are loading two dates from a text file and we want to know how far apart those days are. We need to convert those dates from the text values to numeric ones we can do arithmetic on and only then can we do the difference calculation. Without a preceding load the code will look like this:

You will notice that there is code duplication as we convert both of the dates twice. Whilst I have been very reliably informed (by Henric Cronström) that QlikView with it’s clever caching will not need to calculate the values twice, duplication is a bad thing from a code maintainability point of view.

With a simple preceding load we can remove that duplication and make the code cleaner and more readable. It will then look like this:

You will note that the fields we created in the lower part of the load are then used in the one above.

Also of note here is the asterisk in the preceding load. This is required to pull the fields up from the load below up to what is actually loaded. There are two potential problems you may face here. First if you omit the * then the two date fields will not be in the final data model. The other potential problem is that you can duplicate a field by having a field name used in the succeeding load that is pulled through with a * that is then used again in the preceding load. These can be hard to spot and the error message from QlikView (field names must be unique) does not always point you to the right part of the load script – so be careful.

Generally I would advise against the use of an asterisk (particularly when pulling fields from a database) but in preceding loads they are most useful. Be aware you can also pull fields up explicitly by listing them if you want to take only some fields from your lower LOAD to the preceding one.

Onwards And Upwards

And you don’t need to stop there. If you wanted to have another value calculated on fields derived in your preceding load you can add a preceding load on your preceding load. To take the example of our date interval we could then add another field based on whether a threshold has been breached or not.

Simply stack another load on top of the one before, like this:

There is no limit to the number of levels of load you can have. And whilst it may sound like you are creating a potential confusion, like a BI version of Inception, preceding loads tend to help you to clean and simplify script. You can have expressions that use fields from any of the levels below to create new values.

Advanced Features

Whilst it doesn’t really belong in a back to basics post, I should mention that there are functions you can use in your preceding loads that you may previously have only associated with the first level of load. WHERE, WHILE and GROUP BY are all permissible using values from the load before. You should probably not use these features often, but it is with knowing they are there.

Conclusion

Having preceding loads in your kit bag of QlikView load script code allows you to build complex expressions, whilst keeping your code simple by breaking things up into bite sized chunks. Duplication can be removed and maintainability can be improved. Not bad for a simple little technique.

This is the sixth post in the Back To Basics series. Look out for the next post in the series soon. If you have a suggestion on a topic please let me know in the comments field below.

The post How To Use Preceding Load In QlikView appeared first on .

Since then, I had assumed that this was something everyone knew from their initial QlikView training.  However, on numerous occasions I have used this functionality in the presence of other reasonably experienced QlikView developers and been surprised to find they were unaware.

So… to try and put an end to this here is the simple step you need to take to move objects around inside a chart.

Simply select the chart you want to re-arrange and hold down Shift and Control at the same time.  You will see all the items within the chart get a red border to them:

You can then drag the individual items around inside the chart.  This include the position of the chart itself, the legend and any text in chart items that you have added.  I should warn here than things can sometimes be a bit temperamental –  particularly with objects snapping to the side of the object when you don’t want them to – but with a bit of patience you can usually arrange things how you want.

I don’t expect masses of ‘wow thanks for sharing’ messages for this post (as I got for Hidden QlikView Features) but if you have been stuck not being able to move your Text In Chart then you will know how useful this tip is.

Just one of those features that is not entirely intuitive, like holding the Alt key down to then drag and move a chart or list box object that doesn’t have a caption…

This post is part five in the Back-To-Basics series of posts.

The post Move Items Within A QlikView Chart Object appeared first on .

At the end of last year I uploaded the Quick Intelligence Christmas Card, which included an image of a dashboard with a gauge chart and a pie chart (you can view the card here).  Not something I would look to do in a production document and something that Gysbert Wassenaar quite rightly picked me up on.  He pointed me towards Stephen Few’s Information Dashboard Design – a text I am familiar with and highly recommend.  In fact when I first read the book it caused me to want to go back and revisit some of my previous work to clean it up and make it more clear and concise.  This really is the key to the effective presentation of data.

I have already blogged and uploaded a video tutorial on how you can clean up your charts, which covers chart junk in the form of caption icons, unnecessary shading and colours and a general attention to detail.  Stephen Redmond has also recently blogged on KPI Approaches, advising alternatives to the gauge chart.  I will not attempt to retread either of these paths here, rather suggest the simplest tool for presenting data in QlikView – the humble Text Box.

As I mentioned in my post about caption bars anywhere you see an ellipses (…) in QlikView you can insert an expression.  This is true of a text box.  We can therefore make text boxes a more efficient representation of data.

Take my offending gauge:

This can be adequately be replaced by the following text boxes:

The eagle eyed among you have probably noticed that I needed to put two text boxes in here, as you can not change size, font or style in a single text box.  The background and title are one text box (with alignment top and 6pt text margin) and the percentage is a transparent text box on top.  The expression in our percentage text box is simply: =num(1 – vNice, ‘#,##0.0%’) – the bulk of the calculation being in a variable, and the num function doing the formatting of the number to be a percentage to 1 dp.

We can encode further information into the text box by changing the colour of the background based on the value, mimicking the segment of the gauge the needle falls into:

This colour can be achieved by setting a calculated colour on the text box with the legend  [1].  The code for which is:

=if(vNice > 0.6, $(vGreen), if(vNice > 0.2, $(vOrange), $(vRed)))

Note the use of variables for storing defaults for colours as well as the expressions – this makes code re-use much easier.  I will have to do a future post on the use of variables.

To make the text box even more useful it (or in this case, they) should also have actions attached that allow the user to drill into the detail about that number or fact.  Perhaps in this case a breakdown of which countries have the highest percentage of  naughty children.  This could be achieved by changing the tab that the user was on or using a show / hide variable on a chart object (see my post on Buttons for more information on this).

I used this approach to, I believe, good effect when asked to produce a wall board display of QlikView system status for a customer (who has kindly allowed me to reproduce the dashboard here).  Feeds are brought in from different apps, including CAL usage, the file system and QlikView Distribution logs and aggregated onto a single screen.  The app also fires alert emails when error conditions are encountered:

Most of the objects on this page are text boxes, the exception being a simple table in the top right which has it’s borders and captions removed.  The three main numbers select that type of issue and then go to a tab giving the breakdown of those items.  There are also links to other QlikView apps that contain more information that could be useful in resolving issues.  There are also show conditions on the table and the links, so these don’t display on the wall board machine making it fit on the low-resolution panel on the wall and removing links that can not actually be clicked.

To get back to to the point of this article, think carefully about what object you use to convey information.  Often the most simple approach is the one that will be the most effective.  Keep things simple and your QlikView documents will be all the better for it.

[1] I should also point out that Barry Harmsen mentioned that the level of naughtiness in my dashboard was far too high to still be in the amber segment and I should perhaps be less tolerant of such behaviour.  All I was doing was trying to create an amusing card!

The post Keep QlikView Simple appeared first on .

QlikView QVDs play a crucial part in the majority of QlikView implementations. As such I have blogged about their use many times previously; such as in this post on Incremental Loads and this one on Optimised Loads. Now that I am going Back To Basics in this series of posts I felt it was a good point to go back to first principles with QVDs.

What Is A QVD?

At a basic level a QVD is a flat data file with the extension .qvd. It can store a single table of data (is. each row shares the same column list) and is typically created in the load script of a QVW file. The structure of the file is essentially an XML format, with various bits of header information stored at the top of the file and the rest of the data beneath. One of the beauties of a QVD file is that it is compressed on creation using the same algorithms as QlikView uses to store data in memory – so the files can be incredibly small for the amount of data they contain.  Loading from QVD back into memory is blindingly fast as the format of the file mirrors how QlikView addresses data in RAM.

Why Should You Use QVDs?

One of the points that is often noted when pitching QlikView is that it does not need a data warehouse or predefined data cube to work off. This is completely true, but the requirement to have a logical data layer often remains. This is particularly true if the data being collected is coming from multiple systems or is being distributed via multiple QlikView applications. QVDs can fulfil this requirement admirably. QVDs are also essential when you want to adopt an incremental load strategy.

Personally I would go as far as saying your implementations should always be built on QVDs – except perhaps where the data source is a simple Excel spreadsheet that is stored locally. There are many reasons I would suggest this, some of these are:

• Decoupling data extract from data presentation
• Ability to do parallel data extracts
• Easier unit testing of parts of the load process
• Incremental loads
• Sharing of extracted data between presentation apps
• Ability to scale the solution when data volumes grow
• Ability to delegate responsibility for different parts of the data load to different teams

The question you should be asking when designing your data load strategy is not why you should be using QVDs – but rather are there any reasons why you shouldn’t be using them (the reasons here are very few).

How Do I Create A QVD?

Typically QVDs are created during the execution of your QlikView load script. A STORE statement writes the current contents of a single table in your data model to a file on the disk. The syntax for this is:

STORE TableName INTO ..\Data\FileName.qvd (QVD);

Note that the STORE command can also be used to write data into a comma or tab delimited file – but that is a topic for another blog post.

QVDs can be created during the execution of any QlikView load script, but best practice is to have separate applications which deal solely with the creation of QVD files. This “extraction layer” then handles all interaction with source databases. Depending on the size and complexity of the data being loaded I may create one app for each table being extracted or a single app may create all QVDs for the solution (or anything else  in between). What is important though is that the front end can be reloaded quickly from locally stored data files. This ability to refresh the presentation layer quickly can massively speed up development.

How Do I Use My QVDs?

Once you have created a data layer consisting of a number of QVDs you simply load from each QVD file in the same way you would a CSV or Excel file. The syntax is:

TableName:
LOAD
FieldList
FROM ..\Data\FileName.qvd (qvd);

Note that where with CSV or Excel files the load statement contains a chunk of information about file formats this is not required with a QVD load. You can also use the wizard in the load script by clicking the File button and locating the QVD in the folder browser.

When loading from a QVD you can apply a number of transformations, such as renaming columns, excluding rows and adding derived columns. Be aware though that many of these transformations will cause your QVD load to be non-optimised (which will make the load up to 100 times slower) please see this blog post for more information: QlikView Optimised Loads.

By loading from multiple QVD’s into a single application you can build up your associative data model. This could involve some data from QVDs (perhaps originally from different data sources) and maybe some small lookup type data values from yet another source.

As well as being a very quick way of loading data into QlikView apps QVDs can also be useful for archive – due to the excellent compression which is used in their creation. You could, for example, store dated copies of a data set into QVDs; your front end would then typically work off the latest version, but it would be possible to point it to an archive version of the data if required.

Conclusions

QVD’s are QlikView’s proprietary way of staging and storing data. They can be used to provide a logical data tier in your solution and in some cases could remove the need for a data warehouse. Using QVDs you can share data between multiple applications without having to keep going back to the source data, you can also chunk up the data load process into manageable steps.

If you are presently using QlikView but not using QVDs then I strongly recommend that you consider the advantages of doing so.

The post Introduction to QlikView QVD Files appeared first on .

The Goal

This tip will only take a minute or two to implement, but will give you and your users an immediate visibility of vital information about your document. When it last refreshed, how long it took and how many rules of data were pulled in. The end result will look like this:

Setting Up

To achieve this you will need to set up some variables at the start of your code, to capture the time and to initialise the row counter:

let vLastLoadStart = now(); let vNoOfRows = 0;

You can also capture some other information here. Typically I will set a version number, like this:

let vVersionNumber = 1.03;

Other information can be captured regarding the load here also, such as the machine name and user account used to run the load. Typically I will then set various environment variables after that; such as date formats, colour values and environment variables (such as show/hide states).

At this point I will then tend to our in another tab – to start the load proper.

Keeping Count

To get a row count you will need to add the number of rows in each table to your counter. The code for this is simply:

let vNoOfRows = vNoOfRows + Alt(NoOfRows(‘TableName’), 0);

The Alt statement here ensures your counter is not obliterated if a single table doesn’t load. If you have resident loads or lookup tables in your load script think about whether you want to include these in your count – there’s no right or wrong just as long as you are aware of what it is you are taking.

The other bit of information you need to show the stats that we are after is when the load completed. I will typically put this on an End tab before an Exit Script statement (I will cover the reasons for this elsewhere). The code for this reads:

let vLastLoadEnd = now();

Displaying The Output

The output of this information is done using one of the most n useful and flexible objects QlikView has to offer: the Text Box. Simply place a text box on the front page of your app with the following code in it:

=’Last load date: ‘ & date(vLastLoadStart, ‘DD MMM YYYY hh:mm:ss’) &
‘ Last load duration: ‘ & date(vLastLoadEnd – vLastLoadStart, ‘hh:mm:ss’) &
‘ No of rows: ‘ & num(vNoOfRows, ‘#,##0’) &
‘ Version no: ‘ & vVersionNumber

Typically I will put this at the bottom left corner in a small font, with a grey colour and transparent background. Attention does not need to be drawn to this information – but it is good to be there. If you want to copy and paste a ready made text area you will find them on some of the shared QlikViews I have uploaded to QlikCommunity.

Taking This Further

This simple view of performance is often enough to keep a quick eye on what is happening with your loads. If you need tending of load performance or more granular information (eg. rows and performance of each table in a load) then you need to persist information into QVDs by inserting values into tables during the load – using the AutoGenerate function.

This is the second part in our series of Back To Basics posts, click here to view other posts in this series.

The post How To View QlikView Load Performance appeared first on .

Back To Basics

One of my posts on QlikCommunity recently got an extremely positive response, when I thought I simply was stating the obvious. it then occurred to me that there are a number of things that I do so often that they have become second nature – but they may not seem so obvious to others. That it why I am starting this series of posts – and commencing with the topic from that QlikCommunity post: sorting your QlikView Server folder structure.

Putting The Fault into Default

There are no doubt a number of opinions on the best way to set up your QlikView folder structure (which you are free to share in the comments on this post), but what is universally agreed is that the installation default makes no sense. When you first look at your QMC settings you will see that the folder for publishing QlikView documents is nestled under your Program Files folder and has a handful of demo documents to clutter your Access Point. One of the first things I always do when I go into QMC for the first time is change this, to what I believe to be QlikView best practice.

Mounting Folders In QlikView

For those that are not aware, your published folder configuration can be found in QMC under System / Setup / QlikView Servers / {Your Server Name} / Folders. Here you can set a root folder and a number of sub folders. Ensure that all folders actually exist on your server and that you click the Apply button (I tend to click this twice – just to be sure). Any QlikView document placed in any of these folders or sub folders within them will appear in Access Point (provided the user has the correct AD permissions). For this reason it is important to not mount a folder that sits within another mounted folder – as all documents in that folder will appear in Access Point twice. Given the way that sub folders are included there is only the need to mount a Root folder and then create folders within there. I would however not recommend this. By mounting other folders you have more control over where your documents sit, you can also remove documents from Access Point (perhaps temporarily) without physically removing the files. In this way the Root folder becomes much like the rest of the folders – and I always leave this one empty (you will find system files with a .pgo extension get created in here).

Bring Structure To Your QlikView Folders

As to the structure itself I will typically create a QlikView Documents folder on the data drive of the server (often D:\) and everything else under there. In here I will create folders for the Root folder, Development files, Logs, QVDGeneration, staging Data, SourceData, Images, Themes and also a Published folder – to contain the folders I will mount. The folders within published will change from installation, but will always contain a System folder (for Ops Monitor and other small system apps I use (more on these in a later post)) and then often folders for each area of the business. The end result will look a bit like this: By setting up a sensible structure at the outset much pain can be saved later on. The ability to mount and unmount folders at a granular level makes life a lot easier.

Final Notes

Note that if you have Publisher you will set up the QVDGeneration folder under Distribution settings and the only folders you will mount are Root and those under the Published folder. If you do not have Publisher you will need to mount QVDGeneration also, in order that you can schedule reloads.

You may have noticed I create a Logs folder under QlikView documents, the default logging location also needs to be changed under the QMC settings to point to this folder (on the Logging tab under Server Setup). For more information on setting up mount folders please refer to the QlikView Server manual, search QlikCommunity or feel free to ask a question below.

The post How To Organise Your QlikView Folders appeared first on .