Easy Data Transform v1.39.0 released
Easy Data Transform v1.39.0 is now available for Windows and Mac. Improvements include:
- A new Moving transform for computing a moving average/median/sum/minimum/maximum over an interval, e.g. 7 days.
- Enhancements to the Rename Cols transform that allow you to easily add a suffix or prefix, or make column names unique.
- When you hover over an item in the Center pane, the item and any connections are now subtly highlighted.
- The Summary transform now has additional rows for Min length inc empty and Average length inc empty.
- You can now use the {version} variable anywhere you can use a file name variable.
- You can now edit the comment output to HTML, Markdown, XML and YAML files.
- -0 and -0.0 are no longer output for numerical operations.
- Added warnings for non-numerical values in the Pivot, Stats and Total transforms.
- Fixed an issue where attributes were read from an XML file in a random order.
- Changes to improve consistency between transforms.
- Bug fixes.
Find out more and download the new release from the release page:
This is a free upgrade for everyone with a valid v1 license. If you purchased a v1 license you can have it resent here.
If you don’t have a v1 license you can purchase one here.
Thanks to everyone who sent us bug reports and suggestions for improvements. Please let us know if you have any problems with the new release.
Regression testing
We have spent the last few weeks setting up automated regression tests. These tests use 101 .transform files to generate 859 output files. Here is a screenshot of one of those .transform files, for testing the Stack transform.
This means that we can now run the latest version through a whole battery of tests and check that none of the outputs have changed from the previous version. It also helped us find a number of bugs, edge cases and inconsistencies, which we have fixed in this release.
From this release forward we will be trying hard to ensure that a transform file run from version N will give exactly the same output as when it is run from version N+1 (assuming you haven’t included time or date stamps or changed the locale in Preferences). This means we can move fast and not break things, giving us a really solid basis for moving forward with version 2 in the future.
Note that some of the changes in this release could change the output of existing .transform files.
Column related options
Easy Data Transform tries to be clever enough to update column-related options when columns are added, reordered or renamed upstream. For example, if you use a Case transform to set the URL column to lower case and then insert a Reorder Cols transform upstream to change the column order, it is clever enough to update the Case transform to take account of the column order change.
Issues come when you remove a column upstream, e.g. using Remove Cols or by disconnecting an upstream connection to a transform. We then have to delete all references to that column downstream, for consistency. Even if you later add the column back, all the downstream references are now gone, and you potentially have to go through any downstream transforms and reset all the column related options. Which can be tedious!
In this release we have tried to reduce the chances of resetting downstream column references, by making transform output more robust. For example, the Count transform used to output 2 columns if it had a column selected and 0 columns otherwise. This meant that unselecting the column counted, even temporarily, would nuke all column related options downstream. It now outputs 2 columns in both cases.
However deleting a connection into a transform is still going to delete all column related options downstream. So try to avoid disconnecting transforms!
Note that you insert a transform between 2 existing transforms by selecting a connection and inserting a new transform. You can also delete a transform without deleting it’s connections if it has a single input and output connection. Also, if you accidentally delete a connection, Undo is your friend.
In version 2 we are going to look at ways of restoring previous column-related options when you reconnect a transform.
Windows digital certificate
We have renewed the digital certificate for the Windows version of Easy Data Transform. As this certificate is new to Windows, you could see a few scary warnings. However, if you downloaded it from our website, you can be assured it is safe. Contact us if you have any questions about this.
And finally…
Andy Brice
Oryx Digital Ltd
06-Feb-2023