(disclaimer: this transcript has been automatically transcribed so it may contain errors)

Welcome back to London. Nice to see you all again. Third stream from the UK headquarters of Cubewise. I’m here with Ian. Welcome Ian. Ian’s gonna do an awesome presentation today. I’ve seen a sneak preview. He’s gonna do awesome presentation on testing in TM1, something that’s always an after thought, but it’s super important and Ian’s got a core solution for that.

So yeah, without further ado, Ian. Thank You Simon. Alright, so let’s get started. I’m gonna share my screen here and away we go. We go share that. Okay. Alright, so automated testing in arc. Let’s get started. So I suppose those of you who seen me talk about this kind of stuff before, know I care about testing a lot and know I think it’s really important.

But I guess what, why should we care and why do we care? The important thing to understand with TM1 is it’s, it’s an agile way. We build stuff so it’s really easy to, to build stuff and throw stuff off and get to n to somewhere really, really quick. It’s also really easy to test that as you go along.

So you can build a TI process, you can run some data and you can look in the cube and you can see that’s work that’s done and then you move on to your next thing. So, so testing is a at once in TM1, really, really easy. But also it’s really, really difficult because of the agile nature of building stuff. We can build a lot and then we can kind of forget exactly what we’ve built and, and all of the intricacies in the, the model that we’ve done. So wholesale testing and regression testing is hard to do. There’s no structure or tools in place for us to do that at the moment and we care about that because obviously we want tier one models to be really, really good. So we care about the quality of it and we care about how developers can improve that and how developing and test as they go along and, and build quality models. Before we get started, I’m just gonna do a couple of quick definitions on what I’m gonna be talking about for the rest of the day and then I’m gonna go into a demo of how our compuls can, can help that. So bear with me at the start, it gets a bit wordy at the start, but it’s really simple and we’ll we’ll catch up and start showing some cool stuff in a minute.

Okay, so the first book is a textbook definition of what a unit test is. So I went and grabbed this out of a, a testing textbook. It’s really, really wordy. You can read it, you know, in computer program, blah, blah, blah, blah, blah, blah. Really what that means is, does this thing that I built do what I think it does?

Does it always do what I think it does no matter how many times I do it and does it break something else that somebody else has somebody else has built? Those are the three things. What I’ll focus on with, with the test that I’m gonna show you today. Making sure that the thing works and it works every time. There’s four components of a unit test.

So when you’re building a unit test, you gotta think about the setup. What is the starting environment of this test? So in some cases it can be, there is no data in a queue. In other cases it could be the dimension exists, but it doesn’t have the element that I’m gonna try and and insert into that dimension. It can be very multiple different environments or different setups that you do.

So when you’re building your test, you wanna write the setup so you can control the environment that you are testing and then you do something. And this can be running a process, a chore, entering data into a queue, but there’s, there’s some action that happens and then you’ve got the assertions, which is what most people think of with the test.

The assertions are, which we’ll come into later, the actual, you know, did this thing do what I expect it to do. And then afterwards we clean up. So if you’re doing an example of I’m testing whether or not my TI process in certain element dimension, you might wanna then delete that element at the end of it because you were just using a test element or something like that.

So it’s important to clean up after your test as well and restore the model to the point where you think it should be. Normally while I’m here, I might as well do a textbook definition of an assertion. So again, textbook definition, wordy, wordy stuff, you can read it if you really want to, but it’s pass or fail stuff. At the end of the day, the assertion has to be a tick or a cross. It either passes or it fails. And in what we’re gonna show you today, the assertions can be a number of things, but in the examples that I’m gonna show you today, the assertion is generally gonna be, is this number in this cube? What I expect that number to be. It either is or it isn’t.

So a couple of examples of what what, what I’m gonna run through in a bit. So example of a test for population consolidation element in a in a dimension. So the setup would be we’re gonna create an element in the department dimension called test. So you go in manually put this element in the department dimension and then you run your process and your process is the thing that we’re trying to test.

So the process is supposed to allocate lethal level elements in a dimension to consolidations and the output of that thing that the assertion, the binary draw a false test that you wanna do is, is this test element allocated or a child element of the unallocated consolidation. That’s what the process is supposed to do. That’s what you want test. It’s either a draw or false.

And then the cleanup as the example would be is we’ll delete that element from the department dimension because it’s not supposed to be there. Okay? An example of a general ledger load one, which I think is the actual example we’ve got in coming up is if we clear out the general ledger cube, there’s nothing in that general ledger cube. We start from from fresh empty cube and then we just run our process to load that data.

So something that everybody’s built, lots of times we’re in process to load the data and the expected outcome because we know from the business we’ve, we’ve looked at the raw data before, we know the expected outcome is gonna be revenue in March, 2023, it’s gonna be a hundred thousand and the bank balance in February it can be 9, 9 9. And we’re not gonna do a cleanup because that’s the state of the model that we want in the end is the data is loaded.

Now we know the, the March revenue and the February bank balance because they’re set. We’ve looked at the data source, we know what those things is. When you are in production or development, those numbers might be different. So you might be testing a different number of production or development, but the understanding of the data and the understanding from the business of what those numbers are is what we are testing.

A final one would be something which be testing a rule. So in this instance there is no process to run, we don’t do anything. But the setup to that is we enter a currency attribute and most of you have written a currency calculation before. So the, the currency used by ABC limited, we’ll set that to be GBP and we don’t do anything, we don’t run a process. We just then look at the data in the queue ’cause there’s already data in the queue and we just make sure that that rule is working that local equals GBP. So the number that’s in local equals GBP ’cause it’s a one-to-one currency calculation. If that’s that’s the same we know that rule is working, then we can go back and we can enter USD as the currency tribute for the company. Because it was USD to start with. So we clean ourselves up and we just test by flicking those things on and off. There’s kind of thing that we do manually a lot when we’re building stuff. You’ve got your rules and you’ve got your cube view next to each other and you change the rule, just make sure that the cubeview works, but that’s a one-off.

And what we’re trying to do with what we’ve built in arc is have that so it can be replicable and actionable and everybody can share the tests and we know that those things always passed as we’re building other stuff. Okay, so how can our compulse help in this environment? The first thing is that we’ve tried to do this for a developer. So when you guys are developing and building stuff,

you can also do your testing and write your tests in the same space. So we as developers live in arc. We’d write the tests in Arc as we go along and then we can action those tests again and again and again and we can prove the results and we can run the same tests and we don’t have to come back in the next day, open up the cube view and look at the same cube view just to make sure things happen.

We can just run the tests and know that the tests are looking at the cube views or looking at the dimensions or looking at whatever else things that we’ve, we’ve defined as a test. We can just run it and know that we get green lights or the tests are passing, we’re gonna restore those tests. So if we store the results of those tests, we can track our progress throughout and just make sure we know when the test started to fail or we can use those test results to say we are happy now with what we’ve got. We can promote that to production and we can use pulses to help validate that we’ve tested stuff. I mean that, that’s one of the important things with, with testing as a a former practice manager,

I’d like to know that stuff has been tested and normally that’s a manual process. The, you have to ask the developers if they’ve tested stuff and they would say yes, of course they’ve tested stuff but they can’t show it and prove it. What we’re trying to do is prove that testing to give us the developers, the customers, everybody confidence that something has been tested and we can hand over a testing schedule and say this is what we have tested, this is what works. The other thing that we’re trying to do with this is have non-invasive tests. So we are not writing the test within the code itself, within the TI code itself. We’re not adding objects to tmm one, we’re not adding things in the model itself. We are just having non-invasive tests that we come in, we connect, we execute,

or we read something and then we, we go out. So we are not building stuff in the TM1 model, we’re not adding stuff to an existing TM1 model to do the test. We’re just testing what exists at the moment. Okay, so now time for a demo. I’m gonna jump into Arc, which is a, a new version of Arc that I’ve got installed and we’re gonna run through just what this stuff looks like and how this stuff works in practice.

Okay, so here’s arc. Lemme just refresh that and make sure I’m logged in because we are doing this live. So you can see Arc is running up and what you’ll see here is I’ve got a TM1 server called TM1 Server oh one. I’ve connected Arc to Pulse. So this is Arc plus. This is connected to a pulse environment, which is monitoring Arc, which means I’ve got the new test center, which is in the latest version of Arc Plus, which is available, I think it’s available in quarter four of this year that’s coming out. But here’s a test center and what we’ve got at the moment is the tests. So what you’ll see here is a collection of tests or test collections, unit tests or categories of tests working from right to left.

I’ll just explain what these are categories of just buckets that we put tests in to help us understand what we’re testing. So we, you know, we’ve got a load of tests which relate to bedrock. I just flagged them as bedrock as I create the tests. So I can, I can see and I can categorize these things. The unit tests themselves are, as I say, unit tests. They’re testing a particular thing. So most instances it’s testing a rule or a process or a chore or something like that. But it can be a combination of different things. And then the test collections are how I want to execute those tests. So collection is obviously a collection of tests. So you can see here that in my GL module, my General Ledger module, I’ve got a number of tests that I wanna run as a collection for that module. So in this instance here I’ve got, I’m gonna build the cost dimension. I’m gonna test the build of the cost center dimension and hierarchies. I’m gonna test the build of the account hierarchies just to make sure they’re building and they’re, you know, what they’re supposed to be. Then I’m gonna import some general ledger data. I’m gonna import some KPI assumptions, I’m gonna import some general calculations. And then my final test is, is one that I brought this morning, which is just a test that 2012 is a profit. I’m gonna run those, oops, I should execute that. Okay, off it going.

So you can see it’s, it’s doing the test and it’s doing the, the setup. It’s running the actual test and it’s doing the tear down, tidy in itself afterwards. And it’s giving my results and I’m passing all of my tests. So I know my general ledger module is, is passing as I expected sometimes. So as you can see here, one of my text tests has failed. So my, my test of 2012 profit has failed. So I’m just gonna go into that and we can explain the components of that test with relation to, to what we talked about earlier. So here’s my test. So my test name is test that 2012 is in profit, which I know it is. Otherwise we’ve got got a bit more of a problem than, than just this TI process. And my category is it’s a general ledger test. The execution of that, I’m gonna do a few things. So I’m not gonna do anything in this setup. It doesn’t need to load any data. The data is already loaded from a previous test and that that run perfectly well. I’m not doing anything in the cleanup or the tear down section.

It’s, it’s just testing one thing and it’s just testing a rule. So it’s just testing a rule. There is nothing, no process to run. It didn’t kick off anything, it’s just testing that. But the assertion in there, and there’s only one assertion in this test is it’s assertion a general ledger value, which is, is the name of the assertion that I called this in that assertion it’s a compare cube cell value type.

So I’m looking at the cell value in a cube and I’m comparing that to what I think it should be. The properties are obviously the address of that cell. So that’s cell reference. We know it was looking at 2012. We know it’s looking at profit, we know it’s looking at all those things, which is okay. So that’s the correct address of the cell that I want to test and it’s just greater or or greater than or equal to zero.

It’s in profit, it’s just all I know is it’s something bigger than zero. That’s all I know from the data. I dunno the exact number, I just know it’s something bigger than zero. So we’re failing on that test and I wanna go and find out why. I’m just gonna close that. I’m just gonna open up the general ledge cube. So here’s the general cube, it’s 2012 total Europe, that’s where I was supposed to be looking and operating profit is actually not bigger than zero. Operating profit is now a million less than zero. Now this is quite a common thing that I, you, you find when you’re, you’re building stuff, somebody has amended the process to load debits and credits as opposed to revenue being positive and cost being negative.

This is now showing revenue is negative and cost being positive, common mistake that people make and I wouldn’t have picked that up had I not said it’s greater than zero. It’s obviously the right number, it’s just the wrong, wrong way around. Now we’d have to go and delve in and see why that is, but it’s the wrong number. Lemme go back to my test here and open 2012 is in profit and it goes to my assertion again, edit this one assertion to say actually it’s not greater than or equal to. It should be less than or equal to because we are now building it the other way around. I saved that and I’ll just rerun my tests. Okay, so rerun the same processes that we did before. We’re gonna build the cost center, we’re gonna build the account, we’re gonna import the data, import the assumptions, and then my final test is I’m just gonna test that that one number is now less than zero, not greater than zero. And as you can see now that test passes. Now I need to go and have a word with a business and developer and, and decide whether or not that test is true. Whether or not we are expecting it to be less than zero or greater than zero, but the test now is reputable and I can keep executing that test again and again and again and I come in tomorrow and just make sure that nobody’s changed anything and I’m still getting the same results. Now as I said, we wanna make this easy for developers to do so we in Arc itself and we can build the test and execute the test from Arc.

But a quick way of doing this might be to, to open the cube view. And you see the value here is January is minus 105 6 0. So if I just right click on that cell, we’ve gotta create a new test on the cell itself so I can save a test from that cell. I’m just gonna call this test something else. I’m gonna call this test Jan 2012.

I’m gonna say this is a general ledger test ’cause it is and test again 2012 is less than 100 K. So that’s what my test is gonna be. Again, no setup, no execution, it’s just a rule execute. I’m just making sure that that cell value is the same. I don’t need to run anything before that or after that. But I could choose a process to run if I really wanted to.

Let’s call this, let’s choose the bedrock dimm export. Where’s bedrock Dim export, the, there we go. Bedrock, dim export. Let’s just export a dimension for the sake of it. Export version, export department, and then I’m not going to do anything here. So we’re gonna run a process to do something. And then my assertion, as you know, we came in from the cube cell value so it knows the assertion is gonna be cell value and the assertion properties of that cell are the cell that I came in from. So I, I already chose January, 2012. So we’re at that starting point. We know what that starting point is. All I have to do now is say it should be less than, okay. Okay. And okay, and we can save that test. What did I call that? Just for, so keeping called it January, 2012. Okay, now my general ledger module, I just wanna add that test to my general eligible module list of tests. So I’m just gonna edit that and go to my entries. I’m just gonna find that test that I called it.

So January, 2012, add that to my list of tests, general ledger module. So now we’re getting more granular in the amount of tests that I’m doing for this module and I’m gonna execute that now. Yeah, I don’t wanna execute all of them. So through it’s doing the same test, it’s repeating the test and giving me the same results. So I know this is passing all the time and I’m just adding another test onto it to, to make that test more accurate and more detailed, more I go. As you’ll see with a lot of these execute in a process, there’s, there’s an implied test result in a process with a lot of these. So when we execute a TI process, one of the implied assertions that we’re getting back at the moment is the process executed without any errors.

So the employed variable that we get back from process execution is a tick. That process executed without any errors. We can add different assertions onto those as well to say things like, if I run this process with these parameters, I expect it to execute, to finish with a process break, which is slightly different. The process itself might have failed, but the test is correct because the test is expecting the process to fail.

So the implied variables that we’re getting back from a TI process, more of which come later on in a different presentation, those implied variables are in fact testable themselves as are anything else that you can test within a tier or model. Really cube value is the obvious one to test, but you know, dimension hierarchies actually rules themselves. You know, data within rules itself.

You can write those tests and then test them again and again and again. And as I say, it’s the replicable nature of that. The ability to string test together is trying to, what we’re trying to give developers with this and the ability to, as I say, categorize and store them in pulse so that you can go into pulse later on, view the results of those tests, make sure that testing has been done so that you can control when things move from production. Sorry, from development to production. So there’s lots more tests available for us to do. I could repeat these again, but yeah, just keep running your tests and making sure everything is okay. All right, that’s, that’s what I wanted to show you with a demo.

Okay. And I stop sharing there. Cool. Okay, any questions there? Thanks Ian. I have a question after this for feedback. Yeah, there you go. Feedback. Answer the feedback if you can. That’d be much appreciated. So I have a question. Yeah. Could you combine together stuff like TI and the result of TI with things like did a Python script execute properly and things like that?

Can you like you using TM1py for example? Yeah, I suppose the, The return Values are things that you have in your, your TI process. If a Python file execute properly, you’d have to write back to TM1 to say that that thing execute properly itself. Because Python is seen outside ofTM1 so you’d have to know what the result of that Python thing is going to be to return it back.

So yes possible if you know what that Python file is gonna do. And how about something like if you were exporting your file, could you check that the file, could you check that one of you tests this, it actually creates the file in the first place, like file exists kind of thing? Yeah, so you have file exists in the, the clean down as well.

So the, the the prologue, so the setup of the clean down effectively execute unbound TIs. So you can do anything that you can do in a TI processing prologue, you can check in the, the unbound TI as well. You can do anything you want in there as well. So you can do a file exists and then quit the unbound ti with a process break or a process quit or something like that.

And you know, that unbound ti failed. So your process itself or your testing process itself implicitly fails as well or something like that. So, so yes. Anything that you think you can test in ti code itself or that we can test with the testing methods, which are generally interpreting the model, then yes, testable. Okay, cool. And this is aimed at developers for checking their own work or people who are QA developers?

Well like who, who, who do you think the end user Is? So for, so, so for everyone, I, I suppose the developers when they’re developing stuff are the right person to write the test. Yeah. So you, you know what you’re trying to build and you know that the unit test that you’re trying to do for that kind of stuff,

for an end user, the way end users test system is slightly different. So the end users are coming in from, from the UI or wherever it is that the end users are coming in from and they’re not necessarily testing the execution of a process and the result, that process, they’re testing their experience and things like that. So yeah, so it may be that they look at something and go,

my numbers are wrong, but that’s part of the unit test of what those numbers should be less of the user experience. So, so it’s more end developers for doing their testing for what they, they expect to happen with that unit. Cool. Really cool. And you said it was available this year at some point? Yes. Yes. So I think it’s,

and Vincent might be on the call as well, but I think the next version of Arc Plus is due to be released next quarter. So this will be in it, there’s obviously more going on in the testing area that we’re adding to Arc and Pulse as well. So we’re trying to make a whole testing suite available to them, but, but phase one is this unit testing stuff that we’re releasing there.

Yeah. Cool. Guido just asked, would you run the test only in Dev? Might be the issue might be an issue on Pro and not on Dev. How, how does it work? How would you work from there? So we, we built, we built it so that tests are available to run as long as you can connect to that TM1 instance.

So if I can connect to, to your production TM1 instance, you can run those tests on TM1. Obviously the thing that you want to do is not mess up your data in production, which is why we do the setup and the the clean down as well. Yeah. But yeah, in theory you should be able to migrate something into production, run the same processes and they have the same results.

So you should be able to, to, if you can connect to it, run them on it. But it’s, it should be only the instances and the security that your user has got in arc to run. So if you can’t connect to production, you can’t execute those tests in production. Cool. Vincent confirmed end of the year, so it’s good stuff.

Alright, well I think that’s it. We appreciate it. Thanks. Okay. Is there another question there? I think, I think just Anne Marie’s very excited, so, Okay, there You go. No worries. Thank you. Alright, cool. Thanks Ian. Yep. Appreciate it. Thank you guys. See you again soon. See you soon.