Dynamic UDEs: the Power for Reentrant Processes

Posted on 2016-03-222016-03-23 by mlportersr

If you have an application that you want to construct from multiple parallel processes, a key requirement is signalling – telling the various parts of the application that something important has happened, or is happening, in one of the other processes. When it comes to fulfilling that requirement, a valuable tool to have at hand is the User Defined Event, or UDE. In fact, over the past year this blog has considered a variety of ways to use this tool. However, all these implementations have one thing in common: They all use static UDEs. In other words, the event that will be used to signal a particular occurrence is decided when the code is created and it never changes.

But what if you don’t know until runtime what event you want to use? For example, it is common with reentrant code that you won’t have a fixed set of UDEs because there isn’t a fixed set of VIs that are running. Sometimes you need to be able to send a message to one particular clone. In such a situation, you aren’t just sending general signals, but signals that are unique to a particular instance of a VI. The solution is to use UDEs that are dynamically generated in the same way that the reentrant VIs are.

To demonstrate this technique, the next couple posts will highlight this use case, starting this time with a discussion of some of the design considerations. Our into to this exploration is an application that I was once assigned to maintain.

The Problem Defined

The job asked me to expand an existing application that had been developed by a large LabVIEW consulting firms located here in the US. The problem is that the software wasn’t designed for expandibility. Specifically, a key part of the program was a subsystem that polled a user-defined list of Modbus registers at a rate that was also user-defined. Because the user-defined inputs could change at any time, the decision was made to make the acquisition loops event-driven, and create a separate “metronome” process that would fire an acquisition event at the user defined rate. So far, so good. The real issue is with the implementation of this concept.

Apparently, there was originally only going to be one timed interval but, as you might expect, a requirement was later added to create a second one. To meet this scope change, with as little effort as possible, the decision was made to simply duplicate all the VIs used for the first process while appending “2” to the end of the names – an expedient that is unfortunately common in code developed by “large LabVIEW consulting firms”. To make matters worse, the modularization was poor so the program was basically built around a huge cluster containing literally dozens of references for UDEs, notifiers and queues that ran through nearly every VI in the application.

In the end, the only way to implement the required functionality was to completely redesign and reimplement that portion of the code. The really ironic part is that it took me less time to implement the functionality correctly, than it did to do it poorly the first time. Using this description as a jumping-off point, I obviously won’t be discussing the solution that I implemented for the original application. What we will do is use it as “inspiration” for examining techniques that could cover a wide range of similar requirements.

Getting Moving in the Right Direction

First, we should recognize that while our earlier conversation incorporates a pretty good description of what the code basically needs to do, we do need to flesh it out a bit: At run time, we need to be able to create multiple independently-timed data reads with varying intervals between reads. In addition, the results from these timed acquisitions need to be “published” somehow so they can be used by the rest of the program.

With this broader functional definition in place, we can begin to consider the appropriate API for accessing that functionality. As I have said many times before, one of the corner stones of a good API is the concept of information hiding – the process of deciding what information to expose to the calling code, and what information to keep private. So like a politician running for reelection, our next job is to decide what to hide and where to hide it.

The basic principle in play is to hide any information that the calling code either doesn’t need to know, or which would be counter-productive for it to know. If we think about it, there are exactly three pieces of information that the calling VI actually needs to know in order to define and use an acquisition task:

The Modbus address to read
The interval between reads
How to receive the published data

On the other hand however, there is a (much) large list of things that the calling code doesn’t need to know, among which are things like:

How the Modbus is read
How the timers work
The signalling that the timers use
How many acquisition processes there are
How the timing is implemented
Internal data structures
etc…

Now that we have a handle on what we want to expose – and just as importantly, what we do not want seen – we can start designing the outward interface.

The Outside View

The obvious place to start is with the VI that will configure or setup the polling. Given that we have already decided that we only want to expose two parameters (addresses to read, and the read interval) we can go ahead and create a placeholder VI that provides the appropriate IO, but which is for now empty.

Start Addresses

Note that with the exception of the error cluster, this VI has no other outputs. Remember that all this VI is doing is identifying a group of addresses that some hidden “something” is going to read at the specified interval. Consequently, the only response that this VI can give is whether or not the specification process was successful. The assumption is that other parts of the software will independently report errors that occur during the data reads. In the same way, we are also going to need a VI to tell the “something” that is doing the reading that we are no longer wishing to poll particular addresses.

Stop Addresses

The interface to this VI is even more basic than the one for starting the polling of addresses, and the reason is simple. At this point we don’t care what rate at which a given address is being polled, we just want it to stop. You could argue that knowing the polling interval would make it easier for the code to find the addresses to delete, while that is true, it would also mean that the calling code would have to keep track of the addresses that it is monitoring – which could quickly become an awful lot of redundant information for the caller to remember and manage.

Last but not least, the third interface VI that we need to specify is the data publication mechanism. To keep things simple, we should use a technique that is easy to implement. So I am picking the logical equivalent of the callback techniques apparent in other languages: a User Define Event. For this application, the event will return an array of address/value pairs that the calling application can use as it desires. Note that in this implementation, all the acquisition processes will be sending their data to the same place, so this can be a static UDE.

The Test Application

Finally, while we’re talking about interfaces, let’s also look at the calling application. Because all the “heavy lifting” will be encapsulated in subVIs, the calling code can be very simple. It has buttons for identifying addresses that we wish to poll, addresses we want to stop polling, and stopping the application. To display the results, the application’s front panel incorporates a table that shows the addresses in ascending order, and the last data value acquired for that address.

Main GUI

The block diagram is, likewise, pretty plain. It is event-driven with one event for each of the three buttons on the front panel, plus one to handle the UDE that publishes the data. You can check out its code in the source later.

Crawling Under the Covers

With the front end interfaces thus defined, we now can start thinking about code that will make the interfaces do something useful. The simple part is the UDE for publishing the results because it uses the same technique that we have used many times over the past year. To summarize the implementation, a library named for the event (Publish Data.lvlib) has four subVIs: One (structured as a FGV) for creating/buffering a reference to the event, and one each for registering to receive the event, generating the event, and destroying the event. In addition, it incorporates a typedef that defines the event data.

Publish Data Event Library

The process for managing the addresses to read requires a bit more thought. To begin with, we know that there are only two address management operations: adding and removing addresses. However, we also know that if we are to conform to our API, we need to hide those explicit operations from the calling application. This situation is one of those development scenarios where the words that we use to talk about what we are doing can help or hinder our understanding of what we are trying to accomplish. To see what I mean, let’s consider a similar case that is part of LabVIEW itself: the logic for handling queues.

You may have noticed that with the built-in API you don’t “create” or “destroy” queues. Rather, you “acquire” and “release” references to the queue. While you may wonder what difference this wording makes, we need to remind ourselves that it isn’t simply a matter of an API developer running amuck with a thesaurus. It actually describes a very real and very important distinction. Instead of creating a queue, you are simply telling LabVIEW that you want to acquire a reference to a queue. Now, when you make this request, there are two possible situations:

The queue doesn’t currently exist and LabVIEW needs to create one.
The queue already exists so all you need is to get a reference to the one that is there.

Likewise, you don’t destroy a queue when you are done using it, you simply tell LabVIEW that you have no further need for it by releasing the reference you previously acquired. Because LabVIEW keeps track of how many open references are associated with each queue, LabVIEW can tell when the queue is no longer in use and destroy it automatically. Now consider for a moment the degree to which this hidden functionality simplifies your code. You no longer need to worry about what or who is using the queue, and when it is safe to destroy it. All that potentially complex logic is hidden in the way that the functionality is encapsulated, and the difference in terminology highlights that difference.

As we are designing our API, we need to adhere to the same idea. So instead of “adding” and “removing” address, let’s think about this problem in terms of “starting” and “stopping” acquisition from lists of addresses. To grasp the benefits that we can garner from this change in language, lets consider what actually needs to happen behind the scenes for each of these operations. Just to be clear, this complexity has nothing to do with how we choose to implement the functionality, it is inherent in what we are trying to do. This logic will have to be created regardless of how we structure the code.

Starting Acquisition: This might seem to be pretty easy, but what if users start acquiring data from an address at one rate, but then later changes their mind (or makes a mistake) and starts the same address at a different rate. There is no point to have the same address being read at two different rates. Likewise, it is not clear that this action should be considered an error. Therefore, to do what the user is requesting you first have to remove that address from the process that is currently polling it, and then reassign it to a different (perhaps new, perhaps preexisting) acquisition process. Taking the point further, what if the address you remove from a process is the only address that it is currently polling. Removing that address would leave that acquisition loop with nothing to do, and so we need some way to stop it.

Stopping Acquisition: For its part, stopping the acquisition of addresses can hide some complexity of its own. For example, say the user identifies a list of 4 addresses to be stopped. There is no guarantee that all the addresses are being polled by the same acquisition process – and even if they are all together, we don’t know which process is reading them. This fact implies a need to be able to search all the acquisition processes for a particular address. Plus, as before if we remove all the addresses associated with a particular interval we need to stop that acquisition task.

Remember, this functionality will always be needed, it is simply more robust (and therefore smarter) to hide it from the calling application by encapsulating it in our API.

Getting Down With the Acquisition

To this point we have described the acquisition processes as having two loops: One performs the acquisition and one is a “metronome” function that periodically fires an event that causes the acquisition loop to acquire one scan of all the channels contained in its current configuration. In addition, both of these processes need to be reentrant so multiple copies of each can be launched as needed. Now we need to refine that basic description be specifying the rules that will govern their operation.

First, let’s state that each process is self maintaining both in terms of its own operation and its data. What that requirement means is that each instance of the acquisition process will maintain for itself a list of the addresses that it is polling. Consequently, when a process receives a system message (via UDE) to stop polling on one or more addresses, it will examine its own list of addresses and remove any that are in the “stop polling” list.

Second, we will state that there will only one acquisition process running for each acquisition interval. Hence, if a process receives a system message specifying its own polling interval, it will add those addresses to the list of addresses it is already polling. For example, say there is a process running that is acquiring data from 4 addresses every 1000 msec and it receives a system message that the user wants to start an additional 5 addresses at that same sample interval. The code will add those 5 new addresses to the 4 that it is already polling.

Third, if an acquisition process receives a system start message that does not specify its polling interval, it will prevent duplicate polling by automatically search its configuration for the addresses in the message and delete any that it finds.

Fourth, if after stopping one or more addresses a process finds that its polling list is empty, it will shut itself down by firing an event that is unique to that particular instance.

Fifth, in the event that the user starts addresses for an interval that is not currently running, the logic incorporates a manager function that will start-up a new acquisition process to handle that interval.

Let’s Build Some Code

Finally we are ready to start writing some code to materialize what has to this point been mostly words. A good place to start this work is with the manager VI that will launch new acquisition processes for us. I like this approach because it will give us the opportunity look at how the pieces fit together.

How the Pieces Fit Together

The operational rules we listed earlier provide a number of clues as to where we are going next. To begin with, we talked a lot about messages. This information by itself is enough to let us design and implement the inner workings of the two interface routines we prototyped earlier. They are simply the event generation VIs of two more static UDEs (Start Addresses and Stop Addresses) – and we already know how to build those.

Next, because we have defined what the manager basically needs to do we can create an event-driven shell for it that can respond to two events: Stop Application and Start Addresses. Again, if you have been reading this blog for a while, the Stop Application event is an “old friend” so we will concentrate on the manager’s response to the Start Addresses event. Since this response will vary depending upon whether or not the specified interval already exists, we need to design a way for the code to make that determination, while continuing to bear in mind the principles of information hiding, to wit, the manager doesn’t need to know how the determination was made, just what the result was.

Start Addresses Event Handler

To support this functionality, we will create a VI (called Interval Registry.vi) that the acquisition processes will maintain as they start and stop. This function will support three operations: Check, Insert and Delete. For the Check operation we see here, the routine checks to see if there is already a process running at the indicated sample interval. If there isn’t, the VI returns a false Boolean value that causes the manager to call a launcher VI (Process Launcher.vi) that kicks off all the VIs needed to service the interval. If, however, the interval is already running there is nothing more for the manager to do, so the true case (not shown) does nothing.

Turning our attention now to the VI that launches the new process VIs, we have built this sort of launcher several times before, so we already know its basic structure. What we need clarity on is the details of the data that needs to be passed to the two VIs.

Starting with the simpler of the two (which we will call Metronome.vi) it’s only purpose is to fire an acquisition event at some predefined interval. However, if it is going to stop when it has nothing more to do, it also needs to know what event will tell it to stop. Note that both of these events are specific to each interval that is created. Consequently, they both need to be created on the fly when the acquisition process is launched with their references passed into the new clone as a parameters.

In the same way, looking at the acquisition VI (we’ll call it Modbus Reader.vi) we see that it is going to be receiving the acquisition event that the metronome fires and is also going to need to shut down like the metronome, so it will need to get references to the same to events. The only other messages it will need to receive are the static ones that we have discussed earlier, but because they are static we don’t have to be concerned with them here.

So we add in the event definition logic, and this is what our finished launcher VI looks like:

Process Launcher

The Big Tease

The next thing we need to look into is the VIs that are being launched and the logic that resides inside Interval Registry.vi, but this post is getting long so that will have to wait until next time.

Until Next Time…
Mike…

If the socket fits, wear it…

Posted on 2015-12-072015-12-08 by mlportersr

One of this blog’s recurring themes is the importance of modularity as an expression of the age-old tactic of “divide and conquer”. What is perhaps new (or at least daunting) to some readers is the idea of spreading tasks across not just separate processes on the same computer, but across multiple networked computers. Of course if this strategy is to be successful, the key is communications and to that end we have been examining ways of incorporating remote access capabilities into out testbed application.

Last time out, we implemented the first interface for remote applications to monitor and control our application. That interface took the form of a custom TCP protocol that used packets of JSON data to carry messages over a vanilla TCP connection. I started there because it provides a simplified mechanism for exploring some of the issues concerning basic code structure. Although this interface worked well, and in fact would prove adequate for a wide variety of applications, it did exhibit one big issue. To wit, clients had to be written in a specific way in order to use it. This fact is a problem for many applications because users are growing increasingly reticent about installing special software. They want to know why they need to load special code to do a job? The way they see it, their PCs (and cell phones for that matter) come with a bunch of networking software preloaded on them – and they have a valid point! Why should they have to install something new?

A complete answer to that question is far beyond the scope of this post, but we can spend a few useful moments considering one small niche of the overall problem, and a standardized solution to that problem. Specifically, how can we leverage some of those networking tools (read: browsers) to support remote access to our testbed application? As we have discussed before, the web environment provides ample tools for creating some really nice interfaces. The real sticking point is how that “really nice” interface can communicate with the testbed application. You may recall that a while back we considered one technique that I characterized as a “drop box” solution. The idea was to take advantage of the database underlying a web application by using it to mediate the communications. In other words, the LabVIEW application writes new data to the database and the web application reads and displays the data from the database – hence the “drop box” appellation.

While we might be able to force-fit this approach into providing a control capability, it would impose a couple big problems: First, it would mean that the local application would have to be constantly polling a remote database to see if there have been any changes. Second, it would be really, Really, REALLY slow. We need something faster. We need something more interactive. We need WebSockets.

What are WebSockets?

Simply put, the name WebSockets refers to a message-based protocol that was standardized in 2011 as RFC 6455. The protocol that the standard defines is low-overhead, full-duplex and content agnostic, meaning that it can carry data of any type – even JSON-encoded text data (hint, hint).

An interesting aspect of this protocol is that its default port for establishing a connection is port 80 – the same as the default port for HTTP. While this built-in conflict might be confusing, it actually makes sense. You see when a client initiates an HTTP connection, the first thing it does is pass to the server a number of headers that provide information on the requested connection. One of those headers allows the client to request an Upgrade connection. The original purpose of this header was to allow the client to request an upgraded connection with, for example, enhanced security. However, in recent years it has become a mechanism to allow multiple protocols to listen to the same port.

The way the process works is simple: The client initiates a normal HTTP connection to the server but sets the request headers to indicate that it is requesting a specific non-HTTP protocol. In the case of a request for the WebSockets protocol, the upgrade value is websocket. The server responds to this request with a return code of 101 (Switching Protocols). From that point on, all further communications are made using the WebSockets protocol. It is important to note that while this initial handshake leads some to assume that WebSockets in some ways dependent upon, or rides on top of the HTTP protocol, such is not the case. Aside from the initial connection handshake, the WebSockets protocol is a distinct process that shares nothing with HTTP. Consequently, while the most common application of the technique might be web-based client-server operation, the WebSockets protocol is equally well-suited for peer-to-peer messaging. The only limitation is that one of the two peers needs to be able to respond correctly to the initial handshake.

It is also worth understanding why the basic idea of using Port 80 for the initial connection is significant. A conversation on Stackoverflow gives a pretty good explanation of several issues, but for me the major advantage of using port 80 is that it avoids IT-induced complications. Many corporate IT departments will lock down ports that they don’t recognize. While there are some that try to lock down port 80, it is much less common. Before continuing on, if you’re interested, you also can find the details of the initial handshake here.

The LabVIEW Connection

Ok, so it sounds like WebSockets could definitely have a place in our communications toolbox, but how are we going to take advantage of it from LabVIEW? The answer to that question lies in the work of LabVIEW CLA Sam Sharp. He has developed a set of “pure G” VIs that allows you to implement either side of the connection. Because these are written in nothing but G, there are no DLLs involved so they can run equally well on any supported LabVIEW platform. Making the deal even sweeter, he has documented his code, created a tutorial on them, released his VIs for anyone to use, and all the compensation he requests is “…it would be great if you credit me…”. So, Sam, may you have a million click-throughs.

The following discussion is written assuming Sam’s VIs which I have converted to LabVIEW 2015. One quick note, if you don’t or can’t use the VIPM, you can still use the *.vip file, all you have to do is change the “v” to a “z” and you are good to go. As a first taste of how these VIs work, let’s look (like we did with the TCP example last time) at an over-simplified example to get a sense of the overall logical flow.

For our purposes here, the testbed application will be the “server” so our code starts by listening for a connection attempt on the default Port 80. When it receives a connection, a reference to that connection is passed to a VI (DoHandshake.vi) that implements the initial handshake to activate the WebSockets protocol. Note that a key part of this process is the passing of a couple of “magic strings” between the client and server to validate the connection and protocol selection.

With the handshake completed and both ends of the connection satisfied that the WebSockets protocol is going to be used, the following subVI (Read.vi) reads a data packet from the client that, in our application, represents a data or control request. Next comes the subVI (Write.vi) that writes a response back to the client. Finally the code calls a subVI (Close.vi) that sends a WebSockets command to close the connection, and then closes the TCP connection reference that LabVIEW uses.

Building the Interface

To build this bare logic into something usable, the structure of the server task is essentially identical to that of the TCP process we built last time. In fact, the only difference between the two is ports to which they are listening, and the specific reentrant handlers that they launch in response to a TCP connection. So let’s concentrate on that alternate process. During initialization, the handler calls the subVI that implements the initial handshake.

In addition to the connection reference, this routine also outputs a string that is the URI that was used to establish the connection. Although we don’t need it for our application, it could be used to pass additional information to the server. Once initialization is complete the main event loop starts, but unlike the TCP handler we wrote earlier, it is not based around a state-machine structure.

While we could have broken up the process into separate states, the fact that Sam has provided excellent subVIs implementing the read and write functionality makes such a structure feel a bit contrived – or at least to me it does. When the timeout event fires, the code waits for 500 msecs for the first user data coming from the connection. If the read times-out, the loop waits for another 500 msec and then tries again. This polling technique is important because it allows other things (like the system shutdown event) to interrupt the process. Likewise, because we are waiting for a response that is, at least potentially, coming from a remote computer the polling allows us to wait as long as necessary for the response.

When the request data does arrive, the JSON data string is processed by a pair of subVIs that we originally created for the TCP protocol handler. They create the appropriate Remote Access Commands object and pass it on to the dynamic dispatch VI (Process Command.vi) that executes the command and returns the response. The response data is next flattened to a JSON string and written to the connection. Because the current implementation assumes a single request/response cycle per connection, the code closes the WebSockets connection and the TCP connection reference. However, it would be easy to visualize a structure that would not close the connection, but rather repeat one of the data read commands at a timed interval to create a remote “live” interface.

In terms of the errors that can occur during this process, the code has to correctly respond to two specific error codes. First is error code 56, a built-in LabVIEW error that flags a network operation timeout. Because this is the error that is generated if server hasn’t yet received the client’s request, the code basically ignores it. Second is error code 6066, which is a WebSockets-specific error defined in RFC 6455 to flag the situation where the remote client closes a WebSockets connection. Our code responds by closing the TCP connection reference and stopping the loop.

Testing our Work

Now that we have our new server up and running we need to be able to test its operation. However, rather than creating another LabVIEW application to act as the test platform, I built it into a web application. The interface consists of a main screen that provides a pop-up menu for selecting what you want to do and 5 other screens, each of which focus on a specific control action. As these things go, I guess it’s not a great web application, but it is serviceable enough for our purposes. If you need a great application, talk to Sam Sharp – that’s what his company does.

The HTML and CSS

As I have preached many times before, one of the things that makes web development powerful is the strict “division of labor” between its various components: the HTML defines the content, the CSS specifies how the content should look, JavaScript implements client-side interactivity and a variety of languages (including JavaScript!) providing server-side programmability. So lets start with a quick look at the HTML that defines my web interface, and CSS that makes it look good in spite of me… In order to provide some context for the following discussion, here is what the main screen looks like:

It has a title, a header and a pop-up menu from which you can select what you want to do. As a demonstration of the effect that CSS can have, here’s the part of the HTML that creates the pop-up menu.

<button class="btn btn-default dropdown-toggle" type="button" data-toggle="dropdown">Available Actions<span class="caret"></span></button>
<ul class="dropdown-menu">
  <li><a href="ReadGraphData.html">Read Graph Data</a></li>
  <li><a href="ReadGraphImage.html">Read Graph Image</a></li>
  <li class="divider"></li>
  <li><a href="SetAcquisitionRate.html">Set Acquisition Rate</a></li>
  <li><a href="SetDataBufferDepth.html">Set Data Buffer Depth</a></li>
  <li><a href="SetTCParameters.html">Set TC Parameters</a></li>
</ul>

You’ll notice that pop-up menu is constructed from two separate elements: A button and an unordered list – normally a set of bullet points – where each item in the list is defined as an anchor with a link to one of the other pages. However, as the picture shows, when this code runs we don’t see a button and a set of bullet points, we see one pop-up menu. How can this be? The magic lies in CSS that dramatically changes the appearance of these elements to give them the appearance of a menu. Likewise, some custom JavaScript makes the visually manipulate elements work like a menu. What is very cool, however, is that the resources making this transformation possible are part of a standard package, called Twitter Bootstrap, that is free for anyone to use. In a similar vein, let’s look at the page that displays a plot of data acquired from the testbed application:

At the top of the screen there’s a small form where the user enters information defining the task to be performed, and a button to initiate the operation that the user is requesting. Below that form, is a blank area where the software will draw the graph of the acquired data. Let’s look at two specific bits of HTML, first the code that builds the data entry form…

<form>
  <fieldset class="input-box">
    <legend>View Graph Data</legend>
    <input type="text" class="str-input" id="ipAddr" value="localhost">  Host</input><br>
    <input type="number" class="num-input" id="portNum" value="80">  Port Number</input><br>
    <select id = "targetPlugin">
      <option value = "Sine Source">Sine Source</option>
      <option value = "Ramp Source">Ramp Source</option>
      <option value = "Hen House TC">Hen House TC</option>
      <option value = "Dog House TC">Dog House TC</option>
      <option value = "Out House TC">Out House TC</option>
    </select><label>  Select Target for Action</label><br>
    <input type="button" id="just-submit-button" value="Send Command">
  </fieldset>
</form>

…and now the code that defines the graph:

<div id="container" style="min-width: 310px; height: 400px; margin: 0 auto"></div>

But, something seems to be missing. The first snippet will create data-entry fields and a button, but what happens when the button is clicked? Apparently, nothing. Likewise, the consider the graphing element. We can see how large the area is to be, but where is the data coming from? And where are the graphing operations? To answer those questions, we need to look elsewhere.

The JavaScript

The power behind much of the web in general – and our application in particular – is the interpreted language JavaScript. In addition to being able to access all resources on your computer, JavaScript can interact directly with web pages and their underlying structures. For folks that like to split hairs, JavaScript is “object-based” because it does support the concept of object, but it is not “object-oriented” because it doesn’t explicitly support classes.

More important for what we are going to be doing is that it supports the concept of “callbacks” (read: User Defined Events). In other words, you can tell JavaScript to automatically performs functions when certain events occur. For example, our JavaScript code is going to be interacting with the web page that loaded it, we need to be sure that the page is fully loaded before that program starts. In order to accomplish that goal, the JavaScript file associated with the page includes this structure:

$(window).load(function() {
	...  // a lot of stuff goes here
});

This code creates a callback for the .load() event. The parameter passed to the .load() event is a reference to the function that JavaScript will run when the event fires. As is common in JavaScript, the code declares the function in line so everything between the opening and closing curly brackets will be executed when the event fires. So after declaring a few variables the code includes this:

$("#just-submit-button").click(function(){
  //The code here retrieves all of the input data and formats the request.
  target = $("#targetPlugin").val();
  remAddr = $("#ipAddr").val();
  remPort = $("#portNum").val();
  jsonData = '\"Read Graph Data\":' + JSON.stringify({"Target":target}); 

  // the websocket logic
  wc_connect(remAddr, remPort, parseData);
  wc_send(jsonData);
});

So the first thing the code does when the page finishes loading is register another callback, but this one defines what JavaScript will do when the user clicks the button in the form. The first three lines read the values of the form data entry fields, and the fourth assembles that data into the JSON string that will be sent to the server. The last two lines are the interface to the WebSockets logic. The first of these lines establishes the connection to the server, while the other one sends the command. But what about the response? Shouldn’t there be a line with a command like wc_receive? You really should be expecting this by now: Inside the wc_connect command the code registers another callback to handle the response.

The event (called onmessage) that is tied to this callback fires when a message is received from the server. The code implementing the callback resides in the file websockets.js (in case you’re curious) and its job is to read the JSON response data packet, check for errors, parse the data and generate the output – the graph. The only question now is, “How does it know how to parse the data and generate the graph?” And the answer is (all together now): “There’s another callback!” See the third parameter of wc_connect, the one named parseData? That value is actually a reference to a function contained in the JavaScript code for this particular page, and is an example of how JavaScript implements a “plugin architecture”. So here is how the data parser for this page starts…

var parseData = function(rawData){
  var plotData = JSON.parse(rawData);
  // trim decimal places
  plotData.forEach(function(element, index, array){
    plotData[index] =  Number(element.toFixed(3));
  });

At this point in the process, the data portion of the response is still a string, so to make processing the data easier, we first parse it to convert it into a JSON object. In the case of this particular response, the resulting object is the array of numbers expressed as strings. Really long strings. You see when LabVIEW encodes a number as a JSON string it includes far more digits of precision than are really needed, so forEach element in the array, I convert the value to a number with 3 decimal places. Here’s the rest of the code:

  // logic for drawing the graph
  $('#container').highcharts({
    title: { text: 'Recent Data', align: 'center' },
    subtitle: { text: 'System: '+remAddr+':'+remPort, align: 'center' },
    xAxis: { title: { text: 'Samples' }, tickInterval: 1 },
    yAxis: { title: { text: 'Amplitude' }, gridLineColor: "#D8D8D8" },
    tooltip: { headerFormat: '<small>Sample: {point.key}</small><br>' },
    series: [{ turboThreshold: 0, name: target, data: plotData, lineWidth: 1, marker:{enabled: false}, color: '#000000' }]
  });
}

This is the code that does the plotting, and as we shall see in a moment, this small amount of code produces a beautiful and highly functional chart that displays the values of individual points in a tooltip when you hover over them with the mouse and even provides a pop-up menu that allows you to save the plot image in a variety of image formats. This functionality is possible thanks to a plotting library called Highcharts that uses the structure defined in the HTML as a placeholder for what is going to draw. I have used this library before in demonstrations because in my experience it is stable, easy to use, and very well-documented. I also like the fact that regardless of what kind of plot I am trying to create they have a demo online that gets me about 95% of the way to my goal. Please note that this library is a commercial product, but they make it available for free for “non-Commercial” applications – however even for commercial usage, the one-time license fee is really pretty reasonable. Finally, even though it doesn’t appear that they actively police their licensing with things like crippled versions or the like, if you are using this on a professional project, pay the people. They have certainly earned their bread.

Testing the Pages

So at last we have our server in place and some test web pages (and supporting code) created. We need to consider how to run the web client. Here you have three options: First, you could just double-click the top-level file in Windows Explorer and Windows will dutifully open the file in your browser and everything will work as it should. Second, if you have access to an existing web server you can copy the dozen or so files to it and test it from there. Third, you could create a small temporary server strictly for testing. If you choose that path, a good option is a server called Express.js. As it name implies, it is written in JavaScript, which means it runs under the Node.JS execution engine. You can set one up sufficient to test our current code in about 10 minutes – including the time required to download the code.

The overall test process is similar to what we did to test the custom TCP server last time. The only significant change is the interface. First, test things that should work to make sure they do. Second, test the things that shouldn’t work and make sure they don’t. Here are examples of what you can expect to see on the graphing and image-fetch screens:

Testbed App – Release 20
Toolbox – Release 17
WebSockets Client – Release 1

Big Tease

So what’s next? We have looked at access via a custom TCP interface and the standard WebSockets interface. How about next time, we look at how to do embed this connectivity in a C++ program using a DLL?

Until Next Time…
Mike…

It’s a big interconnected world – and LabVIEW apps can play too

Posted on 2015-11-032015-11-02 by mlportersr

If I were asked to identify one characteristic that set modern test systems apart from their predecessors, my answer would be clear: distributed processing. In the past, test applications were often monolithic programs – but how things have changed! In the past we have talked several times about how the concept of distributed processing has radically altered our approach to system architecture. In fact, the internal design of our Testbed Application is a very concrete expression of that architectural shift. However, the move away from monolithic programs has had a profound impact in another area as well.

In the days of yore, code you wrote rarely had to interact with outside software. The basic rule was that if you wanted your program to do something you had to implement the functionality yourself. You had to assume this burden because there was no alternative. There were no reusable libraries of software components that you could leverage, and almost no way to pass data from one program to another. About all you could hope for were some OS functions that would allow you to do things like read and write disk files, or draw on the screen. In this blog we have looked at ways that our LabVIEW applications can use outside resources through standardized interfaces like .NET or ActiveX. But what if someone else wants to write a program that uses our LabVIEW code as a drop in component? How does that work?

As it turns out, the developers saw this possibility coming and have provides mechanisms that allow LabVIEW application to provide the same sort of standardized interface that we see other applications present. Over the next few posts, we are going to look at how to use incorporate basic remote interfaces ranging from traditional TCP/IP-based networking to building modern .NET assemblies.

What Can We Access?

However, before we can dig into all of that, we need to think about what these interfaces are going to access. In our case, because we have an existing application that we will be retrofitting to incorporate this functionality, we will be looking at the testbed to identify some basic “touchpoints” that a remote application might want to access. By contrast, if you are creating a new application, the process of identifying and defining these touchpoints should be an integral part of you design methodology from day one.

The following sections present the areas where we will be implementing remote access in our testbed application. Each section will describe the remote interface we desire to add and discuss some of the possible implementations for the interface’s “server” side.

Export Data from Plugins

The obvious place to start is by looking at ways of exporting the data. This is, after all, why most remote applications are going to want to access our application: They want the data that we are gathering. So the first thing to consider is, where does the data reside right now? If you go back and look at the original code, you will see that, in all cases, the primary data display for a plugin is a chart that is plotting one new point at a time. Here is what the logic looked like in the Acquire Sine Data.vi plugin.

As you can see, the only place the simulated data goes after it is “acquired” is the chart. Likewise, if you looked at the code for saving the data, you would see that it was getting the data by reading the chart’s History property.

Now, we could expand on that technique to implement the new export functionality, but there is one big consequence to that decision. Approaching the problem in this way would have the side-effect of tying together the number of data points that are saved to the chart’s configuration. Hence, because the amount of data that a chart buffers can’t be changed at runtime, you would have to modify the LabVIEW code to change the amount of data that is returned to the calling application.

A better solution is to leverage what we have learned recently about DVRs and in-place structures to create a storage location the size of which we can control without modifying the application code. A side-effect of this approach is that we will be able to leverage it to improve the efficiency of the local storage of plugin data – yes, sometimes side-effects are good.

To implement this logic we will need three storage buffers: One for each of the two “acquisition” plugins and one for the reentrant “temperature controller” plugin. The interface to each storage buffer will consist of three VIs, the first one of which is responsible for initializing the buffer:

This screenshot represents the version of the initialization routine that serves the Ramp Signal acquisition process. The basic structure of this code is to create a circular buffer that will save the last N samples – where “N” is a reconfigurable number stored in the database. To support this functionality, the DVR incorporates two values: The array of datapoints and a counter that operates as a pointer to track where the current insertion point is in the buffer. These VIs will be installed in the initialization state of the associated plugin screen’s state machine. With the buffer initialized, we next need to be able to insert data. This is typical code for performing that operation:

Because the DVR data array is initialized with the proper number of elements at startup, all this logic has to do is replace an existing value in the array with a newly acquired datapoint value, using the counter of course to tell it which element to replace. Although we have a value in the DVR called Counter we can’t use it without a little tweaking. Specifically, the DVR’s counter value increments without limit each time a value is inserted, however, there is only a finite number of elements in the data array. What we need for our circular buffer is a counter that starts at 0, counts to N-1 and then returns to 0 and starts over. The code in the image shows the easiest way to generate this counter. Simply take the limitless count and modulo divide it by the number of points in the buffer. The output of the modulo division operation is a quotient and a remainder. The remainder is the counter we need.

Modulo division is also important to the routine for reading the data from the buffer, but in this case we need both values. The quotient output is used to identify when the buffer is still in the process of being filled with the initial N datapoints:

During this initial period, when the quotient is 0, the code uses the remainder to trim off the portion of the buffer contents that are yet to be filled with live data. However, once the buffer is filled, the counter ceases being the a marker identifying the end of the data, and it becomes a demarcation point between the new data and the old data. Therefore, once the quotient increments past 0, a little different processing is required.

Once the circular buffer is full, the element that the remainder is pointing at is the oldest data in the array (chronologically speaking), while the datapoint one element up from it is newest. Hence, while the remainder is still used to split the data array, the point now is to swap the two subarrays to put the data in correct chronological order.

Retrieve Graph Images from Plugins

The next opportunity for remote access is to fetch not the data itself, but a graph of the data as it is shown on the application’s GUI. This functionality could form the basic for a remote user interface, or perhaps as an input to a minimalistic web presentation. Simplifying this operation is a control method that allows you to generate an image of the graph and the various objects surrounding it like the plot legend or cursor display. Consequently, the VI that will be servicing the remote connections only needs to be able to access the chart control reference for each plugin. To make those references available, the code now incorporates a buffer that is structurally very similar to the one that we use to store the VI references that allow the GUI to insert the plugins into its subpanel. Due to its similarity to existing code, I won’t cover it in detail, but here are a few key points:

Encapsulated in a library to establish a namespace and provided access control
The FGV that actually stores the references is scoped as Private
Access to the functionality is mediated though publicly-scoped VIs

This FGV is the only new code we will need to add to the existing code.

Adding Remote Control

One thing that both of the remote hooks we just discussed have in common is that they are both pretty passive – or to make this point another way, they both are monitoring what the application is doing without changing what it is doing. Now we want to look at some remote hooks that will allow remote applications control the application’s operation, at least in a limited way.

Since the way the application works is largely dependent upon the contents of the database, it should surprise no one that these control functions will need to provide provisions for the remote application to alter the database contents in a safe and controlled way.

Some Things to Consider

The really important words in that last sentence are “safe” and “controlled”. You see, the thing is that as long as you are simply letting people read the data you are generating, your potential risk is often limited to the value of the data that you are exposing. However, when you give remote users or other applications the ability to control your system, the potential exists that you could lose everything. Please understand that I take no joy in this conversation – I still remember working professionally on a computer that didn’t even have a password. However, in a world where “cyber-crime”, “cyber-terrorism” and “cyber-warfare” have become household terms, this conversation is unavoidable.

To begin with, as a disclaimer you should note that I will not be presenting anything close to a complete security solution, just the part of it that involves test applications directly. The advice I will be providing assumes that you, or someone within your organization, has already done the basic work of securing your network and the computers on that network.

So when it comes to securing applications that you write, the basic principle in play here is that you never give direct access to anything. You always qualify, error-check and validate all inputs coming from remote users or applications. True, you should be doing this input validation anyway, but the fact of the matter is that most developers don’t put a lot of time into validating inputs coming from local users. So here are a couple of recommendations:

Parametrize by Selecting Values – This idea is an expansion on a basic concept I use when creating any sort of interface. I have often said that anything you can do to take the keyboard out of your users’ hands is a good thing. By replacing data that has to be typed with data menus from which they can select you make software more robust and reduce errors. When working with remote interfaces, you do have to support typed strings because unless the remote application was written in LabVIEW, typing is the only option. But what you can do is limit the inputs to a list of specific values. On the LabVIEW-side the code can convert those string values into either a valid enumeration, or a predefined error that cancels the operation and leaves your system unaltered. When dealing with numbers, be sure to validate them also by applying common-sense limits to the inputs.

Create Well-Defined APIs – You want to define a set of interfaces that specify exactly what each operation does, and with as few side-effects as possible. In fancy computer-science terms, this means that operations should be atomic functions that either succeed or fail as a unit. No half-way states allowed! Needless to say, a huge part of being “well-defined” is that the APIs are well-documented. When someone calls a remote function, they should know exactly what is expected of them and exactly what they will get in response.

Keep it Simple – Let’s be honest, the “Swiss Army Knife” approach to interface design can be enticing. You only have to design one interface where everything is parametrized and you’re done, or at least you seem to be for a while. The problem is that as requirements change and expand you have to be constantly adding to that one routine and sooner or later (typically sooner) you will run into something that doesn’t quite fit well into the structure that you created. When that happens, people often try to take the “easy” path and modify their one interface to allow it to handle this new special case – after all, “…it’s just one special case…”. However once you start down that road, special cases tend to multiply like rabbits and the next thing you know, your interface is a complicated, insecure mess. The approach that is truly simple is to create an interface that implements separate calls or functions for each logical piece of information.

With those guidelines in mind, let’s look at the three parameters that we are going to be allowing remote users or applications to access. I picked these parameters because each shows a slightly different use case.

Set the Acquisition Sample Interval

One of the basic ways that you can store a set of parameters is using a DVR, and I demonstrated this technique by using it to store the sample rates that the “acquisition” loops use to pace their operation. In the original code, the parameter was already designed to be changed during operation. You will recall that the basic idea for the parameter’s storage was that of a drop box. It wasn’t important that the logic using the data know exactly when the parameter was changed, as long as it got the correct value the next time it tried to use the data. Consequently, we already have a VI that writes to the DVR (called Sample Rate.lvlib:Write.vi) and, as it turns out, it is all we will need moving forward.

Set Number of Samples to Save

This parameter is interesting because it’s a parameter that didn’t even exist until we started adding the logic for exporting the plugin data. This fact makes it a good illustration of the principle that one change can easily lead to requirements that spawn yet other changes. In this case, creating resizable data buffers leads to the need to be able change the size of those buffers.

To this point, the libraries that we have defined to encapsulate these buffers each incorporate three VIs: one to initialize the buffer, one to insert a new datapoint into it, and one to read all the data stored in the buffer. A logical extension of this pattern would be the addition of a fourth VI, this time one to resize the existing buffer. Called Reset Buffer Size.vi these routines are responsible for both resizing the buffer, and correctly positioning the existing data in the new buffer space. So the first thing the code does is borrow the logic from the buffer reading code to put the dataset in the proper order with the oldest samples at the top and the newest samples at the bottom.

Next the code compares the new and old buffer sizes in order to determine whether the buffer is growing larger, shrinking smaller or staying the same size. Note that the mechanism for performing this “comparison” is to subtract the two value. While a math function might seem to be a curious comparison operator, this technique makes it easy to identify the three conditions that we need to detect. For example, if the values are the same the difference will be 0, and the code can use that value to bypass further operations. Likewise, if the two numbers are not equal, the sign of the result will indicate which input is larger, and the absolute magnitude of the result tells us how much difference there is between the two.

This is the code that is selected when the result of the subtraction is a positive number representing the number of element that are to be added to the buffer.

The code uses the difference value to create an array of appropriate size and then appends it to the bottom of the existing array. In addition, the logic has to set the Counter value point to the first element of the newly appended values so the next insert will go in the correct place. By contrast, if the buffer is shrinking in size, we need to operate on the top of the array.

Because the buffer is getting smaller, the difference is a negative number representing the number of elements to be removed from the buffer data. Hence, the first thing we need to do is extract the number’s absolute value and use it to split the array, effectively removing the elements above the split point. As before, we also need to set the Counter value, but the logic is a little more involved.

You will remember that the most recent data is on the bottom of the array, so where does the next data point need to go? That’s right, the buffer has to wrap around and insert the next datapoint at element 0 again, but here is where the extra complexity comes in. If we simply set Counter to 0 the data insert logic won’t work correctly. Reviewing the insert logic you will notice that the first pass through the buffer (modulo quotient = 0) is handled differently. What we need is to reinitialize Counter with a number that when subjected to the modulo division will result in a remainder of 0, and a quotient that is not 0. An easily derived value that meets that criteria is the size of the array itself.

Finally we have to decide what to do when the buffer size isn’t changing, and here is that code. Based on our discussions just now, you should be able to understand it.

Set Temperature Controller Operating Limits

Finally, there are two reasons I wanted to look at this operation: First, it is an example of where you can have several related parameters that logically form a single value. In this case, we have 5 separate numbers that, together, define the operation of one of the “temperature controller” processes. You need to be on the look-out for this sort of situation because, while treating this information as 5 distinct value would not be essentially wrong, that treatment would result in you needing to generate a lot of redundant code.

However, this parameter illustrates a larger issue, namely that changes in requirements can make design decisions you made earlier – let’s say – problematic. As it was originally designed, the temperature controller values were loaded when the plugins initialized, and they were never intended to be changed during while the plugin was running. However, our new requirement to provide remote control functionality means that this parameter now needs to be dynamic. When confronted with such a situation, you need to look for a solution that will require the least rework of existing code and the fewest side-effects. So you could:

Redesign the plugin so it can restart itself: This might sound inviting at first because the reloading of the operating limits would occur “automatically” when the plugin restarted. Unfortunately, it also means that you would have to add a whole new piece of functionality: the ability for the application to stop and then restart a plugin. Moreover, you would be creating a situation where, from the standpoint of a local operator, some part of the system would be restarting itself at odd intervals for no apparent reason. Not a good idea.
Redesign the plugin to update the limits on the fly: This idea is a bit better, but because the limits are currently being carried through the state machine in a cluster that resides in a shift-register, to implement this idea we will need to interrupt the state machine to make the change. Imposing such an interruption risks disrupting the state machine’s timing.

The best solution (as in all such cases) is to address the fundamental cause: the setups only load when the plugin starts and so are carried in the typedef cluster. The first step is to remove the 5 numbers associated with the temperature controller operating parameters from the cluster. Because the cluster is a typedef, this change conveniently doesn’t modify the plugin itself, though it does alter a couple of subVIs – which even more conveniently show up as being broken. All that is needed to repairs these VIs is to go through them one by one and modify the code to read the now-missing cluster data values with the corresponding values that the buffered configuration VI reads from the database. Said configuration VI (Load Machine Configuration.vi) also requires one very small tweak:

Previously, the only time logic would force a reload of the data was when the VI had not been called before. This modification adds an input to allow the calling code to request a reload by setting the new Reload? input to true. To prevent this change from impacting the places where the VI is already being called, the default value for this input is false, the input is tied to a here-to-fore unused terminal on the connector pane, and the terminal is marked as an Optional input.

Building Out the Infrastructure

At this point in the process, all the modifications that need to be done to the plugins themselves have been accomplished, so now we need is a place for the external interface functionality itself to live. One of the basic concepts of good software design is to look at functionality from the standpoint of what you don’t know or what is likely to change in the future, and then put those things into abstracted modules by themselves. In this way, you can isolate the application as a whole, and thus protect it from changes that occur in the modularized functionality.

The way this concepts applies to the current question should be obvious: There is no way that we can in the here and now develop a compete list of the remote access functionality that users will require in the future. The whole question is at its essence, open-ended. Regardless if how much time you spend studying the issue, users have an inherently different view of your application than you do and so they will come up with needs that you can’t imagine. Hence, while today we might be able to shoe-horn the various data access and control functions into different places in the current structure, to do so would be to start down a dead-end road because it is unlikely that those modifications would meet the requirements of tomorrow. What we need here is a separate process that will allow us to expand or alter the suite of data access and control functionality we will offer.

Introducing the Remote Access Engine

The name of our new process is Remote Access.vi and (like most of the code in the project) it is designed utilizing an event-drive structure that ensures it is quiescent unless it is being actively accessed. The process’ basic theory of operation is that when one of its events fire, it performs the requested operation and then sends a reply in the form of a notification. The name of the notification used for the reply is sent as part of the event data. This basic process is not very different from the concept of “callbacks” used in languages such as JavaScript.

Although this process is primarily intended to run unseen in the background, I have added three indicators to its front panel as aides in troubleshooting. These indicators show the name of the last event that it received, the name of the plugin that the event was targeting, and the name of the response notifier.

The Read Graph Data Event

The description of this event handler will be longer than the others because it pretty much sets the pattern that we will see repeated for each of the other events. It starts by calling a subVI (Validate Plugin Name.vi) that tests to see if the Graph Name coming from the event data is a valid plugin name, and if so, returns the appropriate enumeration.

The heart of this routine is the built-in Scan from String function. However, due to the way the scan operation operates, there are edge conditions where it might not always perform as expected when used by itself. Let’s say I have a typedef enumeration named Things I Spend Too Much Time Obsessing Over.ctl with the values My House, My Car, My Cell Phone, and My House Boat, in that order. Now as I attempt to scan these values from strings, I notice a couple of “issues”. First there is the problems of false positives. As you would expect, it correctly converts the string “My House Boat” into the enumerated value My House Boat. However, it would also convert the string “My House Boat on the Grand Canal” to the same enumeration and pass the last part of the string (” on the Grand Canal”) out its remaining string output. Please note that this behavior is not a bug. In fact, in many situations it can be a very useful behavior – it’s just not the behavior that we want right now because we are only interested in exact matches. To trap this situation, the code marks the name as invalid if the remaining string output is not empty.

The other issue you can have is what I call the default output problem. The scan function is designed such that if the input string is not scanned successfully, it outputs the value present at the default value input. Again, this operation can be a good thing, but it is not the behavior that we want. To deal with this difference, the code tests the error cluster output (which generates and error code 85 for a failed scan) and marks the name as invalid if there is an error.

When Validate Plugin Name.vi finishes executing, we have a converted plugin name and a flag that tells us whether or not we can trust it. So the first thing we do is test that flag to see whether to continue processing the event or return an error message to the caller. Here is the code that executes when the result of the name validation process is Name Not Valid.

If the Response Notifier value from the event data is not null, the code uses it to send the error message, “Update Failed”. Note that this same message is sent whenever any problem arises in the remote interface. While this convention certainly results in a non-specific error message, it also ensures that the error message doesn’t give away any hints to “bad guys” trying to break in. If the Response Notifier value is null (as it will be for local requests) the code does nothing – remember we are also wanting to leverage this logic locally.

If the result of the name validation process is Name Valid, the code now considers the Plugin Name enumeration and predicates its further actions based on what it finds there. This example for Sine Source shows the basic logic.

The code reads the data buffer associated with the signal and passes the result into a case structure that does one of two different things depending on whether the event was fired locally, or resulted from a remote request. For a remote request (Response Notifier is not null), the code turns the data into a variant and uses it as the data for the response notifier. However, if the request is local…

…it sends the same data to the VI that saves a local copy of the data.

The Read Graph Image Event

As I promised above, this event shares much of the basic structure as the one we just considered. In fact, the processing for a Name Not Valid validation result is identical. The Name Valid case, however, is a bit simpler:

The reason for this simplification is that regardless of the plugin being accessed, the datatypes involved in the operation are always the same. The code always starts with a graph control reference (which I get from the lookup buffer) and always generates an Image Data cluster. If the event was fired locally, the image data is passed to a VI (Write PNG File.vi) that prompts the user for a file name and then saves it locally. However, if instead of saving a file, you are wanting to pass the image in a form that is usable in a non-LabVIEW environment, a bit more work is required. To encapsulate that added logic, I created the subVI Send Image Data.vi.

The idea is to convert the proprietary image data coming from the invoke node into a generic form by rendering it as a standard format image. Once in that form, it is a simple matter to send it as a binary stream. To implement this approach, the code first saves the image to a temporary png file. It then reads back the binary contents of the file and uses it as the data for the response notifier. Finally, it deletes the (now redundant) temporary file.

The Set Acquisition Rate Event

This event is the first one to control the operation of the application. It also has no need to be leveraged locally, so no dual operation depending on the contents of the Response Notifier value.

Moreover, because the event action is a command and not a request, the response can only have one of two values: “Update Failed” or “Update Good”. The success message is only be sent if the plugin name is either Sine Source or Ramp Source, and no errors occurs during the update. While on the topic of errors, there are two operations that need to be performed for a complete update: the code must modify both the database and the buffer holding the live copy of the setting that the rest of the application uses. In setting the order of these two operations, I considered which of the two is most likely to generate an error and put it first. When you consider that most of the places storing live data are incapable of generating an error, the database update clearly should go first.

So after verifying the plugin name, the subVI responsible for updating the database (Set Default Sample Period.vi) looks to see if the value is changing. If the “new” value and the “old” value are equal, the code returns a Boolean false to its Changed? output and sets the Result output to Update Good. It might seem strange to return a value to the remote application that the update succeeded when there was no update performed, but think about it from the standpoint of the remote user. If I want a sample period of 1000ms, an output of Update Good tells me I got what I wanted – I don’t care that it didn’t have to change to get there. If the value is changing…

…the code validates the input by sending it to a subVI that compares it to some set limits (500 < period < 2500). Right now these limits are hardcoded, and in some cases that might be perfectly fine. You will encounter many situations where the limits are fixed by the physics of a process or a required input to some piece of equipment. Still, you might want these limits to be programmable too, but I’ll leave that modification as, “…as exercise for the reader.” In any case, if the data is valid, the code uses it to update the database and sets the subVI’s two outputs to reflect whether the database update generated an error. If the data is not valid, it returns the standard error message stating so.

The Set Data Buffer Depth Event

The basic dataflow for this event is very much like the previous one.

The primary logical difference between the two is that all plugins support this parameter. The logic simply has to select the correct buffer to resize.

The Set TC Parameters Event

With our third and (for now at least) final control event, we return to one that is only valid for some of the plugins – this time the temperature controllers.

The interesting part of this event processing is that, because its data was not originally intended to be reloaded at runtime, the live copy of the data is read and buffered in the object-oriented configuration VIs.

Consequently, the routine to update the database (Save Machine Configuration.vi) first creates a Config Data object and then use that object to read the current configuration data. If the data has changed, and is valid, the same object is passed on to the method that writes the data to the database. Note also, that the validation criteria is more complex.

In addition to simple limits on the sample interval, the Error High Level cannot exceed 100, the Error Low Level cannot go below 30, and all the levels have to be correct relative to each other.

Testing

With the last of the basic interface code written and in place, we need to look at how to test it. To aide in that effort, I created five test VIs – one for each event. The point of these VIs is to simply exercise the associated event so we can observe and validate the code’s response. For instance, here’s the one for reading the graph data:

It incorporates two separate execution paths because it has two things that it has to be doing in parallel: Sending the event (the top path) and waiting for a response (the bottom path). Driving both paths, is the output from a support VI from the notifier library (not_Generic Named Notifier.lvlib:Generate Notifier Name.vi). It’s job is to generate a unique notifier name based on the current time and a 4-digit random number. Once the upper path has fired the event, it’s work is done. The bottom path displays the raw notifier response and graphs of the data that is transferred. Next, the test VI for reading the graph image sports similar logic, but the processing of the response data is more complex.

Here, the response notifier name is also used to form the name for a temporary png file that the code uses to store the image data from the event response. As soon as the file is written, the code reads it back in as a png image and passes it to a subVI that writes it to a 2D picture control on the VI’s front panel. Finally, the three test VIs for the control operations are so similar, I’ll only show one of them.

This exemplar is for resizing the data buffers. The only response processing is that it displays the raw variant response data.

To use these VIs, launch the testbed application and run these test VIs manually one at a time. For the VIs that set operating parameters, observe that entering invalid data generates the standard error message. Likewise, when you enter a valid setting look for the correct response in both the program’s behavior and the data stored in the database. For the VI’s testing the read functions, run them and observe that the data they display matches what the selected plugin shows on the application’s GUI.

Testbed Application – Release 18
Toolbox – Release 15

The Big Tease

In this post, we have successfully implemented a remote access/control capability. However, we don’t as of yet have any way of accessing that capability from outside LabVIEW. Next time, we start correcting that matter by creating a TCP/IP interface into the logic we just created. After that introduction, there will be posts covering .NET, ActiveX and maybe even WebSockets – we’ll see how it goes.

Until Next Time…
Mike…

More Than One Kind of Modularity

Posted on 2015-07-132015-07-13 by mlportersr

When learning something that you haven’t done before – like .NET – it’s not uncommon to go through a phase where you look at some of the code you wrote early on and cringe (or at least sigh deeply). The problem is that you are often not only learning a new interface or API, but you are learning how to best use that interface or API. The cause of all the cringing and sighing is that as you learn more, you begin to realize that some of the assumptions and design decisions that you made were misguided, if not flat-out wrong. If you look at the code we put together last time to help us learn about .NET in general, and the NotifyIcon assemble in particular, we see a gold-plated example of just such code. Although it is clearly accomplished it’s original goal of demonstrating how to access .NET functionality and illustrating how the various objects can relate to one another, it is certainly not reusable – or maintainable, or extensible, or any of the other “-ables” that good software needs to be.

In fact, I created the code in that way so this time we can take the lesson one step further to fix those shortcomings, and thus demonstrate how you can go about cleaning up code (of your own or inherited) that is making you cringe or sigh. Remember, it is always worth your time to fix bad design. I can’t tell you how many times I have seen people struggling with bad decisions made years before. Rather than taking a bit of time to fix the root cause of their trouble, they continue to waste hours on project after project in order to workaround the problem.

Ok, so where do we start?

Clearly this code would benefit from cleaning-up and refactoring, but where and how should we start? Well, if you are working on an older code base, the question of where to start will not be a problem. You start with where the most pain is. To put it another way, start with the things that cause you the biggest problems on a day-to-day basis.

This point, however, doesn’t mean that you should just sit around and wait for problems to arise. As you are working always be asking yourself if what you are doing has limitations, or embodies assumptions that might cause problems in the future.

The next thing to remember is that this work can, and should, be iterative. In other words you don’t have to fix everything at once. Start with the most egregious errors, and address the others as you have the opportunity. For example, if you see the code doing something stupid like using a string as a state variable, you can fix that quickly by replacing the strings with a typedef enumeration. I have even fixed some long-standing bugs in doing this replacement because it resolved places where states were subtly misspelled or contained extraneous spaces.

Finally, remember that the biggest payoffs, in terms of long-term benefit, come from improved modularity that corrects basic architectural problems. As we shall see in the following discussion, I include under this broad heading modularity in all its forms: modular functionality, modular logic and modular data.

Revisiting Modular Functionality

Modular functionality is the result of taking small reusable bits of code and encapsulating it in routines that simplify access, standardize the interface or ensure proper usage. There are good examples of all these usages in the application modified code, starting with Create NotifyIcon.vi:

Your first thought might be why I bothered turning this functionality into a subVI. After all, it’s just one constructor node. Well, yes that is true but it’s also true that in order to create that one node you have to remember multiple steps and object names. Even though this subVI appears rather simple, if you consider what it would take to recreate it multiple times in the future, you realize that it actually encapsulates quite a bit of knowledge. Moreover, I want to point out that this knowledge is largely stuff for which there is no overwhelming benefit to be gained from you committing it to memory.

Next, let’s consider the question of standardizing interfaces. Our example in this case is a new subVI I created to handle the task of assigning an icon to the interface we are creating. I have named it Set NotifyIcon Icon.vi:

You will remember from out previous discussion that this task involves passing a .NET object encapsulating the icon we wish to use to a property node for the NotifyIcon object. Originally, this property was combined with several others on a single node. A more flexible approach is to breakup that functionality and standardize the interfaces for all the subVIs that will be setting NotifyIcon to simply consist of an object reference and the data to be used to set the property in a standard LabVIEW datatype – in this case a path input. This approach also clearly simplifies access to the desired functionality.

Finally, there is the matter of ensuring proper usage. A good place to highlight that feature is in the last subVI that the application calls before quitting: Drop NotifyIcon.vi.

You have probably been warned many times about the necessity of closing references that you open. However, when working with .NET objects, that action by itself is sometimes not sufficient to completely release all the system resources that the assembly had been using. Most of the time if you don’t completely close out the assembly you many notice memory leaks or errors from attempting to access resources that still think they are busy. However with the NotifyIcon assembly you will see a problem that is far more noticeable, and embarrassing. If you don’t call the Dispose method your program will close and release all the memory it was using, but if you go to the System Tray you’ll still see your icon. In fact, you will be able to open its menu and even make selections – it just doesn’t do anything. Moreover, the only way to make it go away it to restart your computer.

Given the consequences of forgetting to include this method in your shutdown sequence, it is a good idea to make it an integral part of the code that you can’t forget to include.

Getting Down with Modular Logic

But as powerful as this technique is, there can still be situations where the basic concept of modularity needs to be expressed in a slightly different way. To see such a situation, let’s look at the structure that results from simply applying the previous form of modularity to the problem of building the menus that go with the icon.

Comparing this diagram to the original one from last time, you can see that I have encapsulated the repetitive code that generated the MenuItem objects into dedicated subVIs. By any measure this change is a significant improvement: the code is cleaner, better organized, and far more readable. For example, it is pretty easy to visualize what menu items are on submenus. However, in cases such as this one, this improved readability can be a bit of a double-edged sword. To see what I mean, consider that for the structure of your code to allow you to visualize your menu organization, said organization must be hard-coded into the structure of the code. Consequently, changes to the menus will, as a matter of course, require modification to the fundamental structure of the code. If the justifications for modularity is to include concepts like flexibility and reusability, you just missed the boat.

The solution to this situation is to realize that there is more than one flavor of modularity. In addition to modularizing specific functionality, you can also modularize the logic required to perform complex and changeable tasks (like building menus) that you don’t want to hard code. If this seems like a strange idea to you, consider that computers spend most of their time using their generalized hardware to performed specialized tasks defined by lists of instructions called “programs”. The thing that makes this process work is a generalized bit of software called a “compiler” that turns the programs into data structures that the generalized hardware can use to perform specialized actions.

Carrying forward with this line of reasoning, what we need is a simple way of defining a menu structure that is external to our program, and a “menu compiler” that turns that definition into the MenuItem references that our program needs. So let’s build one…

Creating the Data for Our Menu Compiler

So what should this menu definition look like? Well, to answer that question we need to start with the data required to define a single MenuItem. We see that as a minimum, every item in a menu has to have a name for display to the user, a tag to identify it, and a parent tag that says if the item has a parent item (and if so which item is its parent). In addition, we haven’t really talked about it, but the order of references in an array of menu items defines the order in which the items appear in the menu or submenu – so we need a way to specify its menu position as well. Finally, because in the end the menu will consist of a list (array) of menu item references, it makes sense to express the overall menu definition that we will eventually compile into that array of references as a list (and eventually also an array).

But where should we store this list of menu item definitions? At least part of the to this question depends on who you want to be able to modify the menu, and the level of technical expertise that person has. For example, you could store this data in text files as INI keys, or as XML or JSON strings. These files have the advantage of being easy to generate and are readily accessible to anyone who has access to a text editor – of course that is their major disadvantage, as well. Databases on the other hand are more secure, but not as easy to access. For the purposes of this discussion, I’ll store the menu definitions in a JSON file because, when done properly, the whole issue of how to parse the data simply goes away.

To see what I mean, here is a nicely indented JSON file that describes the menu that we have been working using for our example NotifyIcon application:

[
	{
		"Menu Order":0,
		"Item Name":"Larry",
		"Item Tag":"Larry",
		"Parent Tag":"",
		"Enabled":true
	},{
		"Menu Order":1,
		"Item Name":"Moe",
		"Item Tag":"Moe",
		"Parent Tag":"",
		"Enabled":true
	},{
		"Menu Order":2,
		"Item Name":"The Other Stooge",
		"Item Tag":"The Other Stooge",
		"Parent Tag":"",
		"Enabled":true
	},{
		"Menu Order":3,
		"Item Name":"-",
		"Item Tag":"",
		"Parent Tag":"",
		"Enabled":true
	},{
		"Menu Order":4,
		"Item Name":"Quit",
		"Item Tag":"Quit",
		"Parent Tag":"",
		"Enabled":true
	},{
		"Menu Order":0,
		"Item Name":"Curley",
		"Item Tag":"Curley",
		"Parent Tag":"The Other Stooge",
		"Enabled":true
	},{
		"Menu Order":1,
		"Item Name":"Shep",
		"Item Tag":"Shep",
		"Parent Tag":"The Other Stooge",
		"Enabled":true
	},{
		"Menu Order":2,
		"Item Name":"Joe",
		"Item Tag":"Joe",
		"Parent Tag":"The Other Stooge",
		"Enabled":true
	}
]

And here is the LabVIEW code will convert this string into a LabVIEW array (even if it isn’t nicely indented):

JSON has a lot of advantages over techniques like XML: For starters, it’s easier to read, and a lot more efficient, but this is why I really like using JSON: It is so very convenient.

Starting the Compilation

Now that we have our raw menu definition string read into LabVIEW and converted into a datatype that will simplify the next step in the processing, we need to ensure that the data is in the right order. To see why, we need to remember that the final data structure we are building is hierarchical, so the order in which we build it matters. For instance, “The Other Stooge” is a top-level menu item, but it is also a submenu so we can’t build it until we have references to all the menu items that are under it. Likewise, if one of the items under it is a submenu, we can’t build it until all its children are created.

So given the importance of order, we need to be careful how we handle the data because none of the available storage techniques can on their own guarantee proper ordering. The string formats can all be edited manually, and it’s not reasonable to expect people to always type in data in the right order. Even though databases can sort the result of queries, there isn’t enough information in the menu definition to allow it to do so.

The menu definition we created does have a numeric value that specifies the order of items in their respective menus and submenus. We don’t, however, yet have a way of telling the level the items reside at relative to the overall menu structure. Logically we can see that “Larry” is a top-level menu item, and “Shep” is one level down, but we can’t yet determine that information programmatically. Still the information we need is present in the data, it just needs to be massaged a bit. Here is the code for that task:

As you can see, the process is basically pretty simple. I first rewrite the Item Tag value by adding the original Item Tag value to the colon-delimited list that starts with the Parent Tag. I then count the number of colons in the resulting string, and that is my Menu Level value. The exception to this processing are the top-level menu items which are easy to identify due to their null parent tags. I simply force their Menu Level values to zero and replace the null string with a known value that will make the subsequent processing easier. The real magic however, occurs after the loop stops. The code first sorts the array in ascending order and then reverses the array. Due to the way the 1D array sort works when operating on arrays of clusters, the array will be sorted first by Menu Level and then Menu Order – the first two items in the cluster. This sorting, in concert with the array reversal, guarantees that the children of a submenu will always be processed before the submenu item itself.

Some of you may be wondering why we go to all this trouble. After all, couldn’t we just add a value to the menu definition data to hold the Menu Level? Yes, we could, but it’s not a good idea, and here’s why. In some areas of software development (like database development, for instance) the experts put a lot of store in reducing “redundancy” – which they define basically as storing the same piece of information in more than one place. The problem is that if you have redundant information, you have to decide how to respond when the two pieces of information that are supposed to be the same, aren’t. So let’s say we add a field to the menu definition for the menu level. Now we have the same piece of information stored in two different places: It is stored explicitly in the Menu Level value while at the same time it is also stored implicitly in Parent Tag.

Generating the Menu Item “Code”

In order to turn this listing into the MenuItem references we need, we will pass this sorted and ordered array into a loop that will process one element at a time. And here it is:

You can see that the loop carries two shift registers. The top SR holds a 1D array of strings that consists of the submenu tags that the loop has encountered so far. The other SR also carries a 1D array but each element in it is a cluster containing an array of MenuItem references associated with the submenu named in the corresponding element of the top SR.

As the screenshot shows, the first thing that happens in the loop is that the code checks to see if the indexed Item Tag is contained in the top SR. If the tag is missing from the array it means that the item is not a submenu, so the code uses its data to create a non-submenu MenuItem. In parallel with that operation, the code is also determining what to do with the reference that is being created by looking to see if the item’s Parent Tag exists in the top SR. If the item’s parent is also missing from the array, the code creates entries for it in both arrays. If the parent’s tag is found in the top SR, it means that one or more of the item’s sibling items has already been processed so code is executed to add the new MenuItem to the array of existing ones:

Note that the new reference is added to the top of the array. The reason for this departure from the norm is that due to the way the sorting works, the menu order is also reversed and this logic puts the items on each submenu back in their correct order. Note also that during this processing the references associated the menu items are also accumulated in a separate array that will be used to initialize the callbacks. Because the array indexing operation is conditional, only a MenuItem that is not a submenu, will be included in this array.

Generating the Submenu “Code”

If the indexed Item Tag is found in the top SR, the item is a submenu and the MenuItem references needed to create its MenuItem should be in the array of references stored in the bottom SR.

So the first thing the code does is delete the tag and its data from the two array (since they are no longer needed) and uses the data thus obtained to create the submenu’s MenuItem. At the same time, the code is also checking to see if the submenu’s parent exists in the top SR. As before, if the Parent Tag doesn’t exist in the array, the code creates an entry for it, and if it does…

…adds the new MenuItem to the existing array – again at the top of the array. By the time this loop finishes, there should be only one element in each array. The only item left in the top SR should be “[top-menu]” and the bottom SR should be holding the references to the top-level menu items. The array of references is in turn used to create the ContextMenu object which written to the NotifyIcon object’s ContextMenu property.

What Could Possibly Go Wrong?

At this point, you can run the example code and see an iconic system tray interface that behaves pretty much as it did before, but with a few extra selections. However, we need to have a brief conversation about error checking, and frankly in this situation there are two schools of though on this topic. There is ample opportunity for errors to creep into the menu structure. Something as simple as misspelling a parent tag name could result in an “orphan” menu that would never get displayed – or could end up being the only one that is displayed. So the question is how much error checking do we really need to do? There are those that think you should spend a lot of time going through the logic looking for and trapping every possible error.

Given that most menus should be rather minimal, and errors are really obvious, I tend to concentrate on the low-hanging fruit. For example, one simple check that will catch a large number of possible errors, is looking to see if at the end of the processing, there is more than one menu name left in the top SR – and finding an extra one, asserting an error that gives the name of the extra menu. You should probably also use this error as an opportunity to abort the application launch since you could be left in a situation when you can’t shutdown the program because the “Quit” option is missing.

Something else that you might want to consider is what to do if the external file containing the menu definitions comes up missing. The most obvious solution is to, again, abort the application launch with some sort of appropriate error message. However, depending on the application it might be valuable to provide a hard-coded default menu that doesn’t depend on external files and provides a certain minimum level of functionality. In fact, I once worked on an application where this was an explicit requirement because one of the things that the program allowed the user to do was create custom menus, the structure of which was stored in external files.

Stooge Identifier – Release 2
Toolbox – Release 11

The Big Tease

So what are we going to talk about next time? Well something that I have seen coming up a lot lately on the user forum is the need to be able to work with very large datasets. Often, this issue arises when someone tries to display the results of a test that ran for several hours (or days!) only to discover that the complete dataset consists of hundreds of thousands of separate datapoints. While LabVIEW can easily deal with datasets of this magnitude, it should be obvious that you need to really bring you memory management “A” game. Next time will look into how to plot and manage VLDs (Very Large Datasets).

Until Next Time…

Mike…

Expanding Data Processing Bandwidth — Automatically

Posted on 2015-05-112015-05-11 by mlportersr

Well-written software can typically deal with any performance requirement pretty easily as long as the requirement is constant. It’s when requirements change over time that things can get dicey. For example, if your test system generates a new data packet to process every second and it consistently takes 5 seconds for the data to be processed, a little simple math will tell you how much processing bandwidth you need to create to keep up with the flow of data. But how are you to properly size things when variability is inserted into the process? What if the time between data packets can vary between 100 msec and several minutes? Or what if the data processing time can change dramatically due to things like network traffic?

These are the kind of situations where the processing needs to be more than simply “flexible”, it has to be able to automatically maintain its own operation and reconfigure itself on the fly. To demonstrate one possible implementation of this “advanced” technique, we will build on the simple pieces that we have learned in the past. In some ways, good software design techniques are like Lego blocks. Each one by itself is not very impressive, but when you stick them together, magic happens. But before we can stick anything together, we need to understand…

…what we’re going to do.

You’ll notice that any of the bandwidth management challenges that I mentioned earlier can be addressed by either adding more data processing clones, or removing existing ones that are being underutilized. Consequently, the question of how to implement this self-maintenance functionality really gets down to a matter of how to dynamically manage the number of data processing clones that are currently available — which in turn boils down to answering two very simple questions:

1. How do we know we need more?

Given that the whole point of the exercise is to manage a queue, the current state of that queue will give us all the information that we need to answer this question. Specifically, we can know when more processing bandwidth is needed by monitoring how many items are currently in the queue waiting processing. When the code starts to see the depth going steadily up, it can launch additional processes to handle the data backlog. Of course, this functionality assumes that there is a process that is constantly monitoring the queue and managing that aspect of its operation — which we actually have already in the test code from last week (Data Processing Queue Handler.vi). All we have to do is repurpose this VI to be a permanent part of the final application.

2. How do we know a clone is no longer needed?

The one part of the system that knows whether a clone is being under-utilized is, in fact, the clone itself. As a part of its normal operation, it knows and can keep track of how often is it being used. Having said that, there are (at least) a couple of ways to quantify how much a clone is being utilized. We could, for instance, consider how much time the clone is spending processing data versus how much time it spends waiting to receive data to process. If the utilization percentage drops below a given limit, the clone could then shut itself down. However, for this demonstration, I’m going to use a much simpler criteria that, quite frankly, works pretty well. The code will simply keep track of how many times in a row it goes to the queue and doesn’t find any data.

Code Modifications

Before I start describing the changes that will we will need to make in order to fashion this new ability, I want to consider for a moment the thing that won’t have to change: Queue Test.vi. You might be tempted to say, “Well big deal. All it does is stuff some data into the queue every so often. Who cares if it doesn’t have to change? It’s not even deliverable code”

While that is undoubtedly true, the fact of the matter is that this test routine is important, but not because of what it is or what it does. Rather we care about Queue Test.vi because in our little test environment, it represents the rest of our application — or at least that part of it that is generating data. Consequently, the fact that it doesn’t need modification means that your main application, likewise, won’t need modification if you decide to upgrade from a data processing environment that uses a fixed number of data processors to one that dynamically manages itself.

Data Processing Queue Handler.vi

First, note that previously this routine’s primary job was to simply report how deep the data queue was — a bit of functionality that would likely have not been needed in a real application. Now however, this routine is going to be taking an active part in the process, so I started the modifications by adding the error reporting VI that will transfer errors it generates to the exception handler.

In addition, because the software will initially only start a single data processing clone, I also modified the timeout event handler that performs the VI’s initialization, by removing the loop around the clone launching subVI.

The remainder of the modifications to this routine occurs in the event handler for the Check Queue Size UDE. Previously this event only reported how deep the queue was getting. While it still performs that function, that queue depth information in addition now drives the logic that determines whether we have enough processing bandwidth online.

Note that the queue depth is compared to a new configuration value called Max Queue Size that defines how large the queue can grow before a new data processing clone is launched. Regardless of whether it launches a new clone, event handler calls another new subVI that returns the number clones that are currently running. As you will see in a moment, one of the modifications to the data processing VI is the addition of logic that keeps track of the names of the clones that are running. The subVI that we are calling here returns a count of the number of names that have been recorded so far.

Data Processor.vi

Turning now the data processing code itself, the first stop is in the state-machine’s Initialize state. Here we have all the same logic that existed before, but with a couple minor additions

First, there is a new subVI that registers a clone is starting up. This subVI writes the clone’s name to the FGV that is maintaining the clone count. Second, there is also a new shift register carrying a cluster of internal data that clone will need to do its work. All that is needed during initialization is to set a timestamp value. The Check for Data state is next and it has likewise seen some tweaks — the most significant of which is moving the logic for responding to the dequeue operation into a subVI.

The justification for this move lies in the fact that this logic is now also responsible for determining whether or not the clone is being adequately utilized. As I stated before, each time the clone goes to the queue and comes up empty, the logic will increment a counter that is being carried in the new shift register’s data. If this count exceeds a new configuration value Clone Idle Count, the code will branch to a new state that will shutdown the clone. Likewise, anytime the clone does get data to process, it will reset the count to 0. The changes to the Process Data state, which comes next, are pretty trivial.

All that happens here is that the timestamp extracted from the data to be “analyzed” updates the new cluster data — as well as the indicator on the front panel. Finally, there is the new state: Self Shutdown…

…which simply calls a subVI that removes the clone’s name from FGV maintaining a list of all running clones, and stops the event loop.

Let’s talk about “Race Conditions”

All we have left to do now is test this work and see the differences that it makes, but before we can do that, we need to have a short conversation about race conditions. Very often developers and instructors (myself included) will talk about the necessity of avoiding race conditions. The dirty little secret is that as long as you have multiple things happening in parallel, race conditions will always be present. The real point that these admonitions attempt to make is that you should avoid the race conditions that are unrecognized and potentially problematic.

I bring this point up because as you do the following testing, you may get the chance to see this concept in action. The way it will appear is that the system will launch a new clone immediately after one kills itself off for being underutilized. The reason for this apparent logical lapse is that a race condition exists between the part of the code that is checking to see if another clone is needed and the several places where the clones are deciding whether or not they are being used. There are two causes for this race condition, one we can ameliorate a bit and one over which we have no possible control.

Starting with the cause we can’t control, a simple immutable law of nature is that no matter how sophisticated our logic or algorithms might be, they can not see so much as a nanosecond into the future. Consequently, the first source of a race condition is that when the queue checking logic sees that there are three elements enqueued, it has no way of knowing that a currently active clone will be available in a few milliseconds. While it is true that under certain circumstances it might be possible to provide this logic with a bit of “foresight”, there is no generalized solution to this aspect of the problem. Consequently, this is an issue that we may just have to live with.

The news, however, is better for the second cause. Here the problem is that with all the clones having the same timeout between data checks, it is probable that sooner or later one of the clones is going to become “synchronized” with the others such that it is always checking the queue just after it was emptied by one of the others. However the solution to this problem lies in its very definition. The cure is to see to it that the clones do not have constant timeouts from one check to the next. To implement this concept in our test code I modified the routine that returns the delay to add a small random difference that changes each time it’s called.

The bottom line is that while completely removing all race conditions is not possible, they can be managed such that their impacts are minimized.

New Tests for New Code

The testing of the modified code starts the same as it did before: open and launch Data Processing Queue Handler.vi and Queue Test.vi. The first difference that you will notice is that only 1 clone is initially launched, but at the default data rate, 1 clone is more than adequate.

Now decrease the delay between data packets to 2 seconds. Here the queue depth will bounce around a bit but the clone count will stabilize at 3 or 4.

Finally, take the delay all the way down to 1 second. Initially the clone count may shoot up to 8 or 9, but on my system the clone count eventually settled down to 6 or 7.

At this point, you can begin increasing the delay again and the slowly the clones will start dropping out from disuse. Before you shutdown the test, however, you might want to set the delay back to 2 seconds and leave the code running while you go about whatever else you have to do today. It could be instructive to notice how other things you are doing on the same computer effect the queue operation. You might also want to rerun the test but start Queue Test.vi first and let it run for a minute or so before you startData Processing Queue Handler.vi — just to see what happens.

Further Enhancements?

So we have our basic scalable system completed, but are there things we could still do to improve its operation? Of course. For example, we know that due to timing issues which we can only partially control, the number of clones that is running at one time can vary a bit, even if the data rate is constant. One thing that could be done to improve efficiency would be to change the way clones are handled. For example, right now a data processor is either in memory and running or it is closed. One thing you could do is create a new state that a clone could be in — like loaded into memory, but inactive. You could implement this zombie state by setting the timeout to 0, thus effectively turning off the state machine.

It might also be helpful to change to queue depth limit at which a new clone is created by making it softer. Instead of launching a new clone anytime the queue depth exceeds 3, it might be useful in some situations to maintain a running average and only create a new clone if the average queue depth over the last N checks is greater than 3.

Who knows? Some of you might think of still other modifications and enhancements. The point is to experiment and see what works best for your specific application.

Parallel Data Processing — Release 2

The Big Tease

So what is in store for next time? Well in the past we have discussed how to dynamically launch and use VIs that run as separate processes. But what if the code you want to access dynamically like this happens to exist in a process that you have already compiled into a standalone application? If this application is working you don’t want to risk breaking something by modifying it. As it turns out there are ways to manage and reuse that code as well, even if it was created in an older version of LabVIEW. Next time we’ll start exploring how to do it.

Until Next Time…

Mike…

Finishing the Configuration Management

Posted on 2015-04-202015-04-20 by mlportersr

For the last couple posts we have been looking at how to best utilize object-oriented programming methodologies. In that quest, we have taken as our example the goal of converting the parts of our testbed application that use stored configuration data to a configuration manager based on object classes. In this implementation, the classes represent the various types of potential data repositories.

Maximizing Flexibility by Leveraging Existing Code

When we stopped last time we had created all the basic code infrastructure and all we had to do was construct the dynamic dispatch methods that retrieve the data the application needs. In creating these methods, we have as our guiding principle reducing the amount of code we have to create by reusing as much code as we can. In other words we need to really spend some time thinking about how to structure our code such that it combines maximum reuse with maximum flexibility.

One good way of attaining that ideal is to allow for multiple levels of dynamic dispatch within the same method. Let’s say for the sake of argument that you have a method that will need to be accessible from 10 different subclasses. Furthermore, let’s say that 4 of those subclasses all need to do the same thing, an additional 4 are mostly the same but with a few differences, and the last 2 subclasses require fundamentally different logic to perform the same task. You could implement the logic that is common to the largest group of subclasses (the first 4) in the parent and let the remaining 6 override the parent to define their own solutions. This solution would work, but could potentially result in a lot of duplicated code. It depends on how similar the second group of 4 subclasses are to the first group of 4.

In creating the Initialize New method last week we saw a far better way to optimize the code: For the 4 subclasses that are similar, we could call the parent method in the child (thus taking advantage of that existing code) and then add a little logic to customize the functionality. This solution will work well in many situations, but one area where it will not is in scenarios where the common parts of the code need to pass data to the unique parts.

To address those situations, a very useful solution can be to write the parent method such that it has a subVI encapsulating the similar, but unique bits, that is itself a dynamic dispatch VI. By providing multiple levels of dynamic dispatch, you create a situation that is easy to understand and minimizes duplication of code. In our example, the 4 subclasses would use the parent implementations of the dynamic dispatch VI and subVI. The 4 subclasses that are similar, but a bit different would use the parent method, but override the dynamic dispatch subVI, and the two that are fundamentally different would override the parent method VI itself.

Getting Down to Business

So let’s start looking at what we need to do to finish our conversion — while remembering that most of this discussion will be about the reorganization and repurposing of existing code. For the most part, I won’t be going into how the basic functionality works since we discussed that code when it was originally introduced.

The basic internal structure of our four “blueprint” VIs will be largely similar. We will call the VI that creates the object we want, followed by a dynamic dispatch method that does what we need done. However, note that there isn’t (and shouldn’t be) a direct one-to-one correlation between the blueprint VI and the underlying method. We still need to be looking for ways to minimize redundant code. Back when we were setting up the testbed application originally we considered the need for unstructured configuration values — like you would usually store in an INI file. In fact, we implemented that capability and used it to store the default sample period. Consequently, one of the three methods we end up implementing will be used in a couple of different places now, and will be reusable in the future for other unstructured configuration data. So, let’s start with that one.

Get Misc Setups

Creating this method follows the same procedure as we have used before, except that this method has two input parameters and an output string. So the finished parent method has a front panel that looks like this:

Another difference with this method is that if it is called for a subclass that does not override it, the parent VI does not provide any default functionality. To see what I mean, check out the VI’s block diagram. You see that it does nothing. This structure might seem rather pointless, but in reality it can at times be pretty handy. Say you have a method that is really only needed by one subclass. Without dynamic dispatch you would have to put a case structure around the code so the VI is only called in that one particular circumstance. However, with dynamic dispatch, this selection takes place automatically and without cluttering up the code with case structures.

With the parent VI (Read Misc Settings.vi) created and saved, we can now build the two subclass overrides. Starting with the Text version, to read the data from the INI file, we create an override VI in Config Data_Text.lvclass containing code that uses the built-in configuration VIs to fetch the data:

But wait! This won’t work. You notice that the path to the INI file comes from the class data — which is fine except that we forgot to initialize it. I wanted to highlight this point because it is a very easy, and common mistake to make. Initializing the DVR is not the same as initializing the data in the DVR. All we have to do to fix this memory lapse is modify the class’ version of the Initialize New method, like so:

The subVI with the green banner builds the path to the application’s INI file programmatically. By the way, in case you’re wondering about whether the same issue applies to the other override to this method (the one for databases), the answer is “Yes”. However the solution there is a bit more complex, so we’ll deal with it in a moment.

The other thing to notice about the setup reading method’s code is that if the read doesn’t find the desired value in the INI file, it creates it and gives it a default value. Adding this extra bit is often helpful for recovering from situations where a uses has gotten in and mucked around with the INI file and deleted something they should not have. In any case, to see this code in action, we need to finish up the code in the Get Default Sample Period.vi “blueprint” VI:

You can now run this VI and get a valid result from the INI file. Of course, if you set the configuration data selector value in the INI file to Jet you will get a zero back. This results is because we haven’t provided an override for that subclass yet. Note that you do not get an error because not overriding a parent method is a perfectly valid thing to do.

Updating the Database Initialization

But we need the database data too, so let’s backtrack to the Initialize New VI in Config Data_DB.lvclass. The class data for this subclass is a string that the code will use in one way or another to connect to the database. However, there are three possible sources for this string:

Most ADO interfaces use a rather complex and specialized connection string that can identify logical names, network paths, security parameters and a lot more.
Although Jet does use ADO drivers, its connection string is much simpler. In fact, with the exception of the file path, it can be largely treated as a big string constant.
SQLite (which we aren’t going to support right now, but which exists in the class hierarchy) doesn’t use any ADO parameters. Keeping with its minimalist approach, all it needs is a file path.

So what are we going to do? Well, if you said, this sounds like a job for dynamic dispatch, you’re right! We need to create a method that will derive the correct string depending on which specific subclass is present. So let’s create a virtual folder and a subdirectory named _protected in Config Data_DB.lvclass with Protected access scope. Inside the virtual folder we will create a new VI using the dynamic dispatch template called Get Connection String.vi, and save it in the subdirectory. In addition to the signal IO the template provides, we need to add a string output:

After saving this new the parent class method, we can fill in the overrides. However that code is pretty trivial, mostly scavenged from the original code, and has very little to do with OOP. Consequently, I won’t take time here to highlight it, but feel free to check it for yourself. Then with this work completed, we can finish the initialization code.

Accessing the Database

Although we will be using a Jet database, note that the override for the Read Misc Settings method will reside in the Config Data_DB_ADO.lvclass subclass, not Config Data_DB_ADO_Jet.lvclass. The reason for this placement is that with the exception of the connection string (which we handle during initialization) the query logic is the same for Jet or most other ADO databases. By the way, what would we do if we ever did find a DBMS that needed something different? That’s right, we would just create a subclass for it and override the method in that new subclass.

In any case, most of the code we need now was lifted wholesale from the old configuration library:

And with that we are done with the first of our blueprint VIs. However, this is also the procedure we will follow the VIs to read the error handling parameters, the startup processes and load the machine configurations for each state machine clone. Again, I won’t step through the creation of this code because from the standpoint of this post’s main topic (object oriented code development) there is nothing in these that is any different from the one we did go through. As a friend of mine used to say, “From here on, it’s just a matter of turnin’ the crank.”

Testing

To make the results of these changes easier to verify, I have made slight differences between the configuration in the INI file, and the configuration in the database. Notice that if you start the application with the INI file set to Text, the application launches with only two TC state machines running and the acquisition sample rate defaults to 500 msec. However, a setting of Jet produces the functionality we have seen before (3 state machines and a 1000 msec default sample rate).

Notice also that you have to make the INI file setting changes before starting the application. The logic could certainly have been written to make the settings reconfigurable on the fly, but it would have added another level of complication, so I will leave that as, “…as an exercise for the reader…”.

Before tagging this release of the code in SVN, I also went through the “recycled” code and made sure that the VIs were all saved in their proper locations and all the redundant code had been removed. A project window feature that made the latter task much easier was the ability to right-click on a folder and have LabVIEW list all the files with no callers. One thing to be careful about, however, is deleting “unused” files that are in classes. The logic behind this feature can’t identify VIs that are being dynamically linked — which pretty much describes dynamic dispatch VIs.

Finally, before closings out this post, I want to remind you of something important. I make no claim that this code is the best implementation for all possible situations. Although a lot of it is based on software that I developed for deliverable systems, the point on this blog is to provide you with examples, demonstrations, and models that you can then customize and mold to fit your customers’ specific needs. In music, there is the idea of “variations on a theme”. In fact, in a few cases the variations have become more famous than the original. So take the themes I offer here and feel free to deconstruct, rearrange and reassemble them into something that is new and exciting.

Testbed application Release 15
Toolbox Release 7

The Big Tease

What if I told you that LabVIEW incorporates a feature that can improve the quality of your code and reduce errors by helping to validate your math? Hey, this is a no-brainer! Everybody can use help validating their code. But, what would you say if I told you that most people never use this feature? Crazy…

Until Next Time…

Mike…

Objectifying LabVIEW

Posted on 2015-04-062015-04-12 by mlportersr

I suppose a good place to start this post is with an admission that, in a sense, it is flying a false flag. One way that you could reasonably interpret the title is that in this post I am going to be showing you how to start using objects in LabVIEW. That interpretation is not correct, and the troublesome word is “start”. The fact of the matter is that you can’t use LabVIEW without interacting with objects and many parts of it (think: VI Server) are overtly object-oriented — even without an obvious class structure. The language is built on an objects oriented foundation and so, in a very real way, has been object-oriented since Version 1.

What I am going to be showing you is how to simplify your work by building your own classes. As I stated in the teaser last time, the starting point for this discussion is the recommendation given in NI’s object-oriented training class that you should make your first attempts at using explicit object-oriented technique small, easy to manage subsystems — or put more simply, we need to start with baby steps.

Object-oriented baby steps

OK, so this is the point in the presentation where most presenters hauls out some standard theory, and moth-eaten descriptions of objects and classes — often lifted wholesale from a book on C++ programming. The problem with this approach is of course that we aren’t C++ programmers and the amount of useful information we can draw from an implementation of objects oriented programming that is so fundamentally flawed is minimal at best. The approach I intend to take instead focuses on key aspects of the technique that are of immediate, practical importance to someone who is working in LabVIEW and wants to take advantage of explicitly implementing object-oriented class structures.

A Quick Glossary

The first thing we need is a vocabulary that will let us talk about the topic at hand.

Now be forewarned that some of these definitions may not exactly match what you may read elsewhere, but they are correct for the LabVIEW development environment.

Class — An abstract datatype.
If you think that sounds a lot like the definition of a cluster, you’re right! Due to the way LabVIEW implements object orientation, a class is essentially a very fancy cluster. In fact, when you create a class the first item that LabVIEW inserts into it is a typedef consisting of an empty cluster. Although you don’t have to put anything into the cluster, it provides a place to put data that is private to that class.
Object — An instance of a class.
As with a normal cluster, every instance of a class has its own memory space. Consequently, a class wire is in most ways the same as any other wire in LabVIEW. We are still working in a dataflow environment.
Property — A piece of data that tells you something about the object.
This is why there is a cluster at the heart of the class. You want to put in that cluster information that will describe the object is a way that is meaningful to you application. Because each instance of the class is a separate wire that has its own memory space, the data contained in the cluster describes that particular object.
Method — A VI that is associated with a particular class and which does something useful.
So what do I mean by, “…something useful…”? Well that all depends on the class’ purpose. A the class that is responsible for creating a visual interface might have a method that causes an object to draw itself. While a class that manages the interface to data storage would likely have a method to store or retrieve application data.

From this simple list of words we can begin to see the general shape of the arena in which we will be playing. To recap: A class is a kind of wire. An object is a particular wire. A property is data carried in the wire that describes it in a useful way, and methods use the object data to do something you need done.

Dynamic Dispatch

Now that we have a basic vocabulary in place that lets us talk about this stuff, there are a couple of concepts that we need to discuss. I want to start with this exploration is with the mechanism that LabVIEW uses to call methods. Referred to as dynamic dispatch this feature it is often a source of confusion to developers getting started with object-oriented programming. A good way to come to grips with dynamic dispatch is to compare and contrast it to a feature of LabVIEW with which you may already be familiar: polymorphism.

Polymorphism (from the perspective of the developer using a polymorphic subVI) is the ability of a single functions to adapt to whatever datatype is wired to its inputs. For example, the low-level Add node in LabVIEW is polymorphic. Consequently, it can add scalar numeric if all types, as well as arrays of numerics of varied dimensions, clusters of numerics and even arrays of clusters of numerics.

Of course, from the perspective of the developer creating a polymorphic VI the view is much different. This flexibility doesn’t happen on its own. Rather, you have to create all the individual instance VIs that handle the various datatypes. For example, I often want to know if a value at a specific point in the code has changed from the last time this bit of code executed. So I created a polymorphic VI that performs this function. To create this subVI, I had to write variations of the same basic logic for about a half-dozen or so basic datatypes, as well as a version that used the variant datatype to catch everything else.

Dynamic dispatch (which is actually a form of polymorphism) works much the same way, but with a couple significant differences.

When the decision is made as to which instance VI is to be executed
With conventional polymorphism, the decision of which instance VI to call happens as you wire in the subVI. In the case of my polymorphic subVI, as soon as I wire a U32 to the input, LabVIEW automatically selects the U32 version of the code. However, with dynamic dispatch, that decision gets put off until runtime with LabVIEW making the decision based on the datatype present on the wire as the subVI is called. Of course for that to work, you need a different kind of wire. Which brings us to the other point…
The criteria for choosing between VIs
The wires that conventional polymorphism uses to select a VI all have one thing in common — they are all static datatypes. By that I mean that a wire is a U32, or a string or whatever and it can’t change on the fly. By contrast, with dynamic dispatch, the basis for selection is a wire that is an instance of a class, and the datatype of an object can be dynamic. However this variability is not infinite. A given class wire can’t hold just any object because class structure is also hierarchical.

Say you have a class named Geometric Shapes to Draw. You can define other classes (called subclasses) like Circle or Square that are interpreted by LabVIEW as being more specific instances of Geometric Shapes to Draw objects. Due to this hierarchical relationship, a given wire can be typed as a Geometric Shapes to Draw but at runtime really be carrying a Circle or Square. As a result, a dynamic dispatch VI can call different instance VIs based on the datatype at runtime.

However, one big thing that conventional polymorphism does have in common with dynamic dispatch, is that the power doesn’t come for free. You still have to write the method VIs for dynamic dispatch to call.

Inheritance

Remember a moment ago I referred to class datatypes as being hierarchical? The fancy computer science concept governing the use of hierarchical class structures is called inheritance. The point of this label is to drive home the idea that not only are subclasses logically related to the classes above them in the hierarchy, but these so-called child classes also have access to the properties and methods contained in their parent classes. In other words they can “inherit” or use data and capabilities that belong to their parents.

Handled properly, inheritance can significantly reduced the amount of code that you have to write. Handled poorly, inheritance can turn an otherwise promising project into a veritable train wreck. Which brings up our last point…

Proper Organization

Although organization isn’t really a feature of object-oriented programming, it is never the less critical. The simple fact of the matter is that while a disorganized, undisciplined developer might be able to get by when working in conventional LabVIEW, introducing the explicit use of classes can result in utter chaos. Of the real object-oriented failures that I have seen over the years, they all shared a lack of, or inconsistent, organization.

So what sort of organizational things am I talking about? Well it’s a lot of the same stuff that we have talked about before. For a more general discussion of the topic you can check out a post that I wrote very early on titled, Conventional Wisdom. What I want to do right now is highlight some of the points that are particularly important for object-oriented work.

The two main conventions (directory structure and file naming) go together because the point of one is to mirror the other. But rather than simply list some rules, I’ll demonstrate how this works. To start, I will create a directory that is named for the class hierarchy that I will build inside it. So if the point of this class hierarchy is, for example, to update my program’s user interface, I would call the directory something obvious like GUI Update. Inside this directory I would then create the top-level class with the file name GUI Update.lvclass. At this time I will also create a couple subdirectories (_subVIs and _typedefs) that I know I will undoubtedly be needing. Finally, I have learned over the years that being able to tightly control access to VIs is very important, so I will also create at this time a project library named GUI Update.lvlib and put into it the top-level class and a virtual folder called _subclasses with its access scope set to Private.

So the parent class is set up, but what about the subclasses? I simply repeat the pattern. Let’s say the GUI Update class has subclasses for three types of controls that it will need to update: Boolean, Digital and Cluster. I create subdirectories in the parent directory that are named for the subclass that will go into each, and hierarchically name the three subclasses GUI Update_Boolean.lvclass, GUI Update_Digital.lvclass, and GUI Update_Cluster.lvclass. I am also careful to remember to add the subclass files to the _subclasses virtual folder in the library, edit their icon overlays, and set their inheritance correctly — which is to say, identify their parents. Note that while the hierarchical naming structure doesn’t automatically establish correct inheritance, this convention does make it easier to visualize class relationships in the project file.

And so I go building each layer in my class hierarchy. With each new subclass I continue the same pattern so if I eventually want to find, say a subVI associated with the class GUI Update_Digital_Unsigned Word.lvclass, I know I will find it in the directory ../GUI Update/Digital/Unsigned Word/_subVIs.

Having a pattern to which you stick relentlessly — even one as simple as this one — will save you immeasurable amounts of time.

Creating the Blueprint

The next thing I do when creating a class hierarchy (but the last thing I want to talk about right now) is how the rest of the application will interface with my new GUI Update class. This is where the access scope we have been so careful to create comes into play. In the top-level class I always create a group of VIs that have their access scope set to public. These interface VIs form the totality of the external interface to the class hierarchy and so include the functions that define what the application as a whole needs GUI Update to do for it. The logical implications of this interface layer is why I sometimes call this step in the process, “Creating the Blueprint”.

In addition to providing a very clean interface, another advantage of having this “blueprint” is that if you ever need to expand your stable of subclasses, these interface VIs will serve as a list of functions you need to support in the new subclass — or at least a list of functions that you should consider implementing in the new subclass. To see what I mean, consider that the scenario we have been discussing is actually drawn from an application I created once. The list of public interface VIs was really very short: There was a method that read a value from a remote device and wrote it to the GUI object, one that looked for control value changes to write them to the remote device, and one that allowed the calling application to set control specific properties.

Of these, all GUI objects had to implement the first one because even the controls needed to be updated once a second. The reason for this constraint was that the remote device could also be reconfigured from a local interface and the LabVIEW application needed to keep itself up to date. However, the second interface method was only applicable to controls. Finally, the third interface method was implemented very rarely for the few subclasses that needed it.

What’s up next?

We have just about run out of space for this installment, but you may have noticed that something is missing from this post: Any actual LabVIEW code. Next time we will correct that sad situation by considering how to apply these principles to the creation of a class hierarchy that provides a common mechanism for storing and retrieving program data and setup parameters that works the same (from the application’s perspective at least) regardless of whether the program is interfacing with a database or text files.

Until Next Time…

Mike…

Building a Subpanel-Based User Interface

Posted on 2015-02-092015-03-14 by mlportersr

Having discussed some of the main issues related to incorporating modularity into your application’s GUI, we are now ready to look at how to tweak what we have already built into an application that incorporates those ideas. From the user’s standpoint, the program will operate very much as it did before, but with no data transfer hassles: there’s not any data transfers!

Let’s See Them Work!

To see how subpanels work we will modify our existing testbed application. However, these modifications will only require minor changes to the application. Remember, it is not an accident that this sort of modification can be accomplished without a major rework of the code. This sort of adaptability results from a conscious decision to use techniques and methods that are inherently adaptable. Always be thinking ahead…

And if you do think about what we are doing here you realize the required modifications will be minor in scope because there really isn’t very much in the code that fundamentally cares how the interface is structured. In fact, there are only two places that will need to be changed. The GUI needs to know what to do when a different screen is selected — this logic needs to change. Likewise, the acquisition processes (which previously ran unseen in the background) now need to be prettier, but on the whole they will be doing less than they did before.

Making Background Task Presentable

So let’s start by turning the background tasks into something we would want the world to see on our application’s main interface. It only takes a glance to notice that the existing graphs are too small, so the first step in the transformation is to resize the graphs on our three data sources and make them all the same size as the existing graph in the GUI. In addition, because all three processes will be going into the same subpanel, we also need to make sure the front panels of the three acquisition processes are all the same size: slightly larger than the graph. While we are doing cosmetic things, we also want to make sure that the front panels are the same color as the main GUI.

The functional modification needed to make these VIs live happily in a subpanel is to go into their VI properties and in the Window Appearance section, turn off both scrollbars.

We now turn to the code changes. In order to put a VI into a subpanel you write a reference to the VI to a subpanel control method called Insert VI. The thing is, retaining references to background processes that you launch can get tricky. This point is especially true when you need to launch multiple instances of a reentrant VI. Although the testbed isn’t (currently) using reentrant processes, the software does support it so we need to think about how to get references to the processes that will be going into the subpanel. One simple way to accomplish this goal is to have the processes “publish” a reference to themselves that can be used elsewhere in the code. To implement this approach, I created a buffer that has two public interface VIs. One appends a reference to the buffer contents and one reads the buffer contents. Note that we won’t take the time to look at the code for these functions here because we have examined this structure before.

To use these VIs, we put an instance of the insert VI at the very beginning of the call chain in each of the acquisition processes. Like so:

The only other changes we need to make to the acquisition logic both relate to the timeout value for the loop. We no longer need the acquisition to wait before starting so a single keystroke will change the “-1” timeout in the initialization logic back to “1”.

Likewise, we don’t need the Change Source event right now, but we may need it again (soon) so we’ll leave the event handler in place, but modify the code so it doesn’t do anything.

…and that is all we have to do to the acquisition processes.

Adding the Subpanel to the GUI

With the acquisition processes modified, all we have to left to do is change the front panel control on the GUI, and alter what happens when you make a Data Source selection.

Changing the front panel control consists of simply removing the graph and replacing it with a subpanel. One thing you need to think about is the size that you are going to make the subpanel. I have found through experience that you get a better looking result if you make the subpanel a skosh bigger than the front panel of the VIs that you are putting in it. For the purposes of this discussion a “skosh” is an empirically determined constant equal to 4 pixels, so make the subpanel 4 pixels taller and wider than the acquisition process front panels.

You’ll note that when you place a subpanel on a front panel, LabVIEW also installs an invoke node for it on the block diagram, we’ll use that in a little bit. The only other thing you need to do that is related to the graph is remove the VIs implementing the update UDE, and delete the associated event handler.

Note: Normally I would put a screenshot here illustrating the change, but in this case I can’t think of a good way of showing something which is no longer present, and isn’t being replaced by something…

Concerning the Data Source value change event, this is where we get to use the invoke node that LabVIEW created when you instantiated the subpanel. Remove the VI for firing the UDE, and wire in the invoke node in its place. As you can see the node has one input that is a reference to the VI that is to appear in the subpanel. Now, our technique for getting that reference is remarkably similar to what we did to get the data value for the UDE. The difference is that instead of selecting one element from an array of strings, the same array index node is now selecting one element from an array of VI references — references that come from the read VI for the reference buffer we discussed earlier.

Running the Result

If you run the application that we have created, you will notice that (with the exception of a couple points) the operation is very similar to the way it worked in the last release. As you make selections, you can observe that the data displayed still changes as it did before. But if you watch carefully, you will notice that the plugins continue to acquire data even when you aren’t watching them. Moreover, if you go back to a screen after having not looked at it for few seconds, you will notice that it will initially show the previous data and them update all the “missing” datapoints at once. This might seem strange, but it is the side effect of a very good thing that LabVIEW does for you.

Knowing that updating screens that aren’t currently visible can use a lot of computer resources unnecessarily (especially if they have charts on them), LabVIEW keeps track of front panels that are visible and only takes the time to update the data displays if the screen is visible. The delayed update that you see happening all at once is the result of LabVIEW realizing that the front panel is now visible and sending all the updates to the chart at once. On simple controls and indicators, these updates typically happen so fast that you have to really be looking for them, but I wanted you to see the effect.

Note that in the previous paragraph I did not refer to front panels that are “open”. The thing is, not all screens that are open are visible. For example, VI’s running in the background (like the data acquisition processes used to be) are open but Hidden, i.e. not visible. Conversely, the front panels of VIs in subpanels are visible, but they aren’t open. In any case, here’s the updated code.

Testbed application — Release 9

Project Toolbox — Release 4 (No Change)

Before we move on to the next thing I want to cover, I’m going to hang-out with subpanels for one more post. Specifically, designing VIs in subpanels to be completely autonomous processes is a very powerful technique, but sometimes a good way to foster reusability is to have the main GUI provide some standardized, reusable resources that the various subpanel plugins can use. To cover that situation, next time I’ll look at ways to manage shared resources like buttons and menus from subpanels.

Until next time…

Mike…

Modularization – It’s not just for Block Diagrams (Front Panels can play too!)

Posted on 2015-02-022015-05-10 by mlportersr

At the very end of my last post I suggested that perhaps the best way to display user data on your application’s GUI is to avoid sending data to the GUI in the first place. Of course the big question is how the GUI can display data that you don’t send to it?

The programmatic slight of hand that explains this paradox is a feature of LabVIEW called a subpanel. To understand how subpanels fit into the overall LabVIEW “ecosystem”, let’s review for a moment. As anyone who has used LabVIEW for more than 5 minutes will recall, every VI you create has 3 main parts:

The Front Panel — This part provides the VI with a user interface for interacting with the routine.
The Block Diagram — This window is where you put the graphical source code that describes what you are trying to do.
The Connector Pane/Icon — This part allows you to call your VI from the block diagram of another VI.

Now all programming environments have places to enter source code and methods for calling or reusing other code, but the Front Panel is really unique to LabVIEW. With most languages, you have to write code to create the GUI, but with LabVIEW, the GUI is part of the code. Just as calling one VI from the block diagram of another VI is a way to modularize your program’s logic, so subpanels provide a way to modularize your program’s user interface by incorporating the front panel of one VI into the front panel of another VI.

But Why Subpanels?

To be honest, there are other techniques for “modularizing” user interfaces within LabVIEW, and we’ll take a quick look at a couple — though to be honest we should call the result of one technique “pseudo modularity” because while the interface looks modular, it’s really not.

First we want to consider XControls. A couple years ago at NIWeek, I presented a session on how to use XControls, so it’s a topic with which I am familiar. XControls approach the question of GUI modularization by providing a mechanism for creating custom front panels controls that implement custom functionality not found in standard controls. Although they are very powerful, they have some issues:

Problems with XControls

Complexity: XControls are difficult to create and require both greater skill on the part of the developer and advanced knowledge of how LabVIEW works internally. As you can imagine, an XControl of even modest complexity can take a long time to develop. The result is that the functionality they encapsulate must be sufficiently reusable to justify the effort required to create the XControl.
Can Complicate Debugging: One of the “interesting” aspects of using an XControl is that they are almost always running. Of course, when you think about it, this makes sense. A control obviously has to exhibit some useful behavior at run time. However, during development they must also be able respond to changes that the developer is making. For an XControl this requirement means that, on some level, it is running as soon as you drop it down on a front panel. While this fact obviously complicates the process of debugging the XControl itself, it can also adversely effect the debugging of the program you are trying to create — especially if the XControl uses some of the same libraries that you use in creating your program. You can see limited access to subVIs because they are being called in the XControl and so are already running, or are at least reserved for execution.
Support is Limited: Actually, “stinks on ice” is probably closer to the truth. Documentation is poor, examples are practically nonexistent, and because NI considers it an advanced topic, tech support is often no real help.

Tab Controls — Just Say “No”

This technique is the one I described as “pseudo modularity”. As a control metaphor, the tab is familiar to anyone who has been around computers for a while. However, operating system designers have been moving away from tab interfaces for a variety of reasons. To be fair, I would like to start this section with some of the positive things about tab controls, but to tell the truth I can’t think of any…

Problems with Tab Controls

Complexity: While a tab control might help visually organize the 30 or 40 controls and indicators on your front panel, the block diagram can rapidly become a mess as you try to deal with the functionality associated that many terminals. Moreover, because the tabs hide complexity, they increase the likely that your front panel will have 30 to 40 controls and indicators.
No Reusability: Let’s say you create really nice a user interface on one tab of your application. Now let’s further postulate that the interface is so good that another internal customer wants you to build them an application that uses the same interface. How do you do it? Well, with tabs, your option is to cut and paste the controls and then rebuild the logic on the block diagram.

Of course that’s just the beginning of the challenges. You still have to verify that the code works the same in the new context, and you have to hope that the new program’s timing doesn’t effect how the interface works.
Compatibility Problems: It is unfortunately not uncommon to see posts on the forum where someone is wanting to know why common controls and indicators don’t work as expected when they are on tab controls. Such questions are not uncommon because, to be perfectly frank, LabVIEW has a long history of tab controls causing a variety of screen update and performance problems.
Confused VI Server Interface: Normally when working with controls and indicators the terminals of which are on your block diagram, finding there references programmatically is easy. With a reference to the VI’s front panel you can get an array of the controls on the front panel, and the reference to the control will be in that array.

But a tab control complicates the whole process. First you get a reference to the tab control, from that you get an array of references to the pages in the tab control, and once you have found the right page you can get the array of control references containing the control reference you want.

A Good Solution for GUI Modularity

But there’s another way to answer the question: “But Why Subpanels?”: the advantages that subpanels offer. But please note that you can only expect to realize these advantages if you use subpanels properly — which is to say, VIs appearing in the subpanels are autonomous processes that encapsulate the logic for a single operation and, when necessary, display the results of those operation on their own front panels.

Subpanel Advantages

Subpanels Simplify Code, Reduce Memory Footprint and Improve Efficiency: When using subpanels you are able to realize these benefits because the minimal code required implement them is almost always smaller than the code you take out that implemented the redundant communication and display formatting operations. In addition, remember that because you are reducing the data manipulation the code is performing, you are also reducing the memory (and CPU) needed to manage it.
Subpanels Make Code Robust: When designing a process VI that will run in a subpanel, it is easier to create highly cohesive code with very low coupling between it and the remainder of the application.
Subpanels Generalize GUI Structure: Working with subpanels inherently puts you into a mode where you start thinking about the GUI at a higher level. One side-effect of thinking in this way is that so much application-specific logic is moved into subpanels that, in the end, the GUI itself doesn’t know or care what application it is running.

This point is particularly interesting because the interface could get so generic that it could be divided up into separate dedicated areas like this:

Normally the subpanel frames would not be visible, and there would be no annotations in each section, but then the front panel would be basically blank, and very uninteresting to look at. To get an idea of what could go in each section, consider these descriptions. Remember, not all of these sections would necessarily be on every screen, and their sizes can be adjusted to fit requirements.

Header: The intent of the header subpanel is to show VIs that present information that is unlikely to change as a function of the screen being shown in the body. This area could include things like the current time, the operator who is logged in, total operating hours or the name of the display.

Footer: This subpanel extends across the entire bottom of the screen. It could hold VIs that provide things like a status display, or a scrolling alarm status list. Like the header, this section ideally wouldn’t change during the course of program execution.

Body: The application’s main display VIs would go in this subpanel.

Sidebar: The VIs providing this section’s contents of could include things like navigation buttons, or supplemental information that is related to the main display.

If you have ever done any web development, this basic layout might look a little familiar, which is appropriate since I “appropriated” the idea. One of the key concepts of the web is to separate content from where and how the content is displayed to the user. This approach in LabVIEW is a (small) step in that direction with the subpanels providing the display structure.

So combine these advantages these with a host of others, and you can begin to get a feel for why I consider subpanels to be among the top 5 features added to LabVIEW — ever.

Let’s See Them Work!

To get our first taste of subpanels in action, we won’t go to the extent of creating a fully abstracted interface like we just saw. Rather, I will show how to manage one subpanel — and you can expand it from there, however this post is already getting long, so we’ll have to wait until next time to look at that code.

Until next time…

Mike…

A “Finished” Testbed Application

Posted on 2014-10-302015-08-28 by mlportersr

As I’ve worked on the past several posts, I had a very specific milestone in mind. You see, I am wanting to use this LabVIEW blog to demonstrate and explain a lot of things that I have learned over the years, but the proper presentation of those topics typically requires a certain amount of pre-existing infrastructure.

That is what I have been doing through the preceding posts — building the conceptual basis for a testbed application that embodies what we have discussed so far. Having this infrastructure is critical for thorough, disciplined learning because without it you are left with example code that may demonstrate one point well, but violates a dozen other principles that are standard “best practices”.

Having said that, however, remember the quotation marks around the word Finished in the title. The marks are there to highlight the fact that, as with all LabVIEW applications, this testbed will never really, and finally finished. In fact, it is planned that it will get better over time, and support more functionality. The best we can say is that the testbed is done for now.

Come next week though, who knows? There’s still a lot to do.

The Testbed Project

Like the UDE template, the current version of the testbed and all its associated VIs is available through the website’s subversion repository at:

http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 1

You can use the web client as you did before, but considering the number of files, you will find it to be less time-consuming (and error prone) to use a dedicated client application like TortoiseSVN on Windows. Regardless of how you get the testbed project onto your computer, the first thing that you see with any application is the project file that is its home. When you open testbed.lvproj note that it embodies two of the conventions that I use on a regular basis.

There are virtual folders for two types of items — both of which are contained in LabVIEW libraries:
The _ude folder contains the libraries associated with the two UDEs that are currently being used. The _libraries folders contains the two libraries embodying the main functional areas that the application currently uses. The Configuration Management library houses the files that the application uses to fetch configuration information (right now it only needs to read the launch processes). The Startup Processes library contains the VIs that will be launchable when the application starts, and the process-specific VIs that they use. Finally, the Path Utilities library contains the VIs that the code uses to locate the internal project home directory in an executable.
The launcher VI is directly in My Computer, along with its INI file, testbed.ini.
As I have discussed elsewhere, this naming convention simplifies a variety of tasks such as locating the application INI file.

Looking at the VIs

The remainder of this post will look at the VIs used in creating the testbed application. Right now the discussion will be at a rather high-level as we will be digging into them on a more detailed basis in the future. With each VI, I also list a few of the future topics that I will use that VI to demonstrate. If you would like to see others, let me know in the comments.

Testbed.vi

Here we see pretty much what you would expect. As discussed in the previous post, the code starts by calling a subVI that stores the launcher’s path to a FGV that will make it available to the rest of the application. It then calls the subVI that reads the INI file to get the list of processes to launch.

The array of process descriptions thus retrieved is passed to the main loop that processes each element individually. The LabVIEW documentation does a good job of describing asynchronous call-and-forget so I won’t repeat it here. Instead I will point out the 1 second delay that follows the dynamic call logic. Often times, a new process starting up has a lot it needs to do. This delay gives the new process a 1 second window where the launcher is not doing any potentially time-consuming tasks — like loading into memory the next process it will launch. The delay may not be technically necessary, but I have found that it can sometimes make things run a bit smoother.

Finally, you may be wondering, why there is logic to explicitly close the launcher window? Isn’t there a VI property that tells LabVIEW to automatically close the window if it was originally closed? Well, yes there is, and you would think that it would handle this situation. In fact, it does — but only in the development environment. When you build the application into an executable, the auto-close doesn’t work for the top-level VI window, or at least doesn’t with LabVIEW 2014.

Future Enhancements

Showing the “busy” cursor
Enhancing user feedback
How to launch other standalone executables
How to launch VIs that need input parameters

Startup Processes.lvlib:Acquire Data.vi

This is the VI that the original producer loop turned into. The most noticeable change has been the addition of an event structure that both allows the process to respond to a application stop event, and ( via the timeout event) provides the acquisition timing.

Future Enhancements

Handling errors that occur during initialization
Turning acquisition on and off
Changing acquisition parameters
Correcting for timing “slips” when using the timeout event

Startup Processes.lvlib:Display Data.vi

This VI is the new consumer loop. This VI retains the event loop it had before, but adds an event for stopping the application.

Future Enhancements

Using subpanels to generalize your interface code — and avoid tab controls
Implementing dialog boxes that don’t stop everything while they are open.
The advantages of the “System”-themed controls

In addition to these VIs, there’s no rule that says there can’t be more — like say for handling errors. You may also need additional processes for handling tasks that have differing timing requirements, or communicate through different interfaces. We’ll look at those too.

But for now, feel free to poke around through the code, and if you have any questions ask in the comments. Also try building the application and perhaps even the installer.

Until next time…

Mike…

The Problem Defined

Getting Moving in the Right Direction

The Outside View

The Test Application

Crawling Under the Covers

Getting Down With the Acquisition

Let’s Build Some Code

How the Pieces Fit Together

The Big Tease

Share this:

What are WebSockets?

The LabVIEW Connection

Building the Interface

Testing our Work

The HTML and CSS

The JavaScript

Testing the Pages

Big Tease

Share this:

What Can We Access?

Export Data from Plugins

Retrieve Graph Images from Plugins

Adding Remote Control

Some Things to Consider

Set the Acquisition Sample Interval

Set Number of Samples to Save

Set Temperature Controller Operating Limits

Building Out the Infrastructure

Introducing the Remote Access Engine

The Read Graph Data Event

The Read Graph Image Event

The Set Acquisition Rate Event

The Set Data Buffer Depth Event

The Set TC Parameters Event

Testing

The Big Tease

Share this:

Ok, so where do we start?

Revisiting Modular Functionality

Getting Down with Modular Logic

Creating the Data for Our Menu Compiler

Starting the Compilation

Generating the Menu Item “Code”

Generating the Submenu “Code”

What Could Possibly Go Wrong?

The Big Tease

Share this:

…what we’re going to do.

1. How do we know we need more?

2. How do we know a clone is no longer needed?

Code Modifications

Data Processing Queue Handler.vi

Data Processor.vi

Let’s talk about “Race Conditions”

New Tests for New Code

Further Enhancements?

The Big Tease

Share this:

Maximizing Flexibility by Leveraging Existing Code

Getting Down to Business

Get Misc Setups

Updating the Database Initialization

Accessing the Database

Testing

The Big Tease

Share this:

Object-oriented baby steps

A Quick Glossary

Dynamic Dispatch

Inheritance

Proper Organization

Creating the Blueprint

What’s up next?

Share this:

Let’s See Them Work!

Making Background Task Presentable

Adding the Subpanel to the GUI

Running the Result

Share this:

But Why Subpanels?