One Generator.exe for multiple users?

system · November 10, 2008, 8:46pm

Hi guys,

So as I add more customers to the HaveAByte.com hosted AyaNova service, each one requires their own Generator service to be set up and working. Unfortunately this comes at quite a cost: 33 MB of RAM per instance, which is quite a lot.

I think this could be solved in one of two ways:

A special behavior of the Generator added where it could read a configuration file that points to a list of config.txt files. On each pass (which I believe is on 5 minute intervals), it would do it’s work for each of those configuration files. This seems like a pretty simple change, basically wrapping the main loop in another loop that executes the main loop once for each config.txt file it is pointed to. This way, one Generator.exe could handle as many systems as necessary.
Assuming the Generator always does a check when it is first started up, and assuming there is a way to know that it has completed it’s work (such as by checking the Event Viewer), I could solve this on my end. I’m thinking I could create my own service that would launch each Generator in turn, wait for it to do its work, then shut it down, and launch the next one. When they’re all complete (say this takes 45 seconds), it would start again in 265 seconds (thus maintaining the integrity of the 5 minute loop, more or less.)

Thoughts? This would really help scale the application when there are several low-traffic users on a server that has tons of speed and bandwidth but somewhat limited RAM (like mine!)

Thanks,

Erik

JohnDev · November 25, 2008, 9:29am

HaveAByte (11/10/2008)
2. Assuming the Generator always does a check when it is first started up, and assuming there is a way to know that it has completed it’s work (such as by checking the Event Viewer), I could solve this on my end. I’m thinking I could create my own service that would launch each Generator in turn, wait for it to do its work, then shut it down, and launch the next one. When they’re all complete (say this takes 45 seconds), it would start again in 265 seconds (thus maintaining the integrity of the 5 minute loop, more or less.)

Erik

Hi Erik, as discussed offline we are looking into this for the next release. You could do what you describe however the generator process only logs exceptions not sucessful generations if I recall correctly.

If you are going to go through that much trouble it would be far easier to whip up a little .net program to do the generation any way you want, the actual generation is all handled automatically and requires only two lines of code.

Remember that anything that any of the AyaNova programs and addons can do you can do through code since they are all built on the publicly available business object library that comes with AyaNova already installed.

This is allthe code you need to generate in c#, however you can use any .net language (i.e. vb.net, java.net c++.net what have you)the process is almost identical:

using GZTW.AyaNova.BLL;
class SomeClass{

public static void DoGenerate()

{
try{
AyaBizUtils.Initialize();
AyaBizUtils.Login("username","password");

AyaBizUtils.TG=true;//tell ayanova biz library you are about to generate
GenProcessPM.GeneratePMWorkorders();//generate PM workorders if any are due
GenProcessDeliveries.DeliverNotifications();//process and deliver notifications
}
catch(Exception ex)
{
throw;//ex.Message contains the problem if there was one here for logging or what have you
}
finally
{
AyaBizUtils.TG=false;//Tell the biz object library you are done generating
}

}
}

The actual call to Initialize is where it reads the config.txt file to find the connection to the database so if, after every run through you were to substitute or rewrite the config.txt to another database and call initialize again and login with proper credentials for the next databaseyou could cycle through all the databases automatically.

JohnDev · February 2, 2009, 6:40pm

HaveAByte (11/10/2008)Hi guys,

So as I add more customers to the HaveAByte.com hosted AyaNova service, each one requires their own Generator service to be set up and working. Unfortunately this comes at quite a cost: 33 MB of RAM per instance, which is quite a lot.

Hi Erik, just working on some performance and memory optimization stuff for v5 and came across this again and it didn’t sound right at all so I double checked and v4 AyaNova generator uses around 6mb of private non shared memory per instance, not 33mb.

I suspect you’re getting that figure from Windows Task Manager which does not show the amount of memory that process has held on to and cannot be shared, it shows a different value basically any memory that the process has touched and a very large chunk of memory that the .net memory manager has spoken up for but has not actually held, it’s thinking ahead for performance reasons and so it speaks up for a big chunk of memory, however that memory is free for anyone else who needs it, it’s just putting a light hold on it in case the application quickly needs it and generator does not need anything like that amount.

It is not an accurate figure if you are determining how much ram you need because the great majority of that figure is memory that is actually free to go to anyone else who want’s it should memory get low enough that it would matter in the operating system.

You can get an accurate figure for ram that is held and not usable by any other process by checking the Private Bytes performance monitor counter or use a tool like process explorer and check the performance properties private bytes which I just did for v4 generator and it gave a value of 5,912kb.

.net memory management is a complex subject, there is a fairly clear analogy here that explains reserving the big chunk of memory and why you can’t go by the task manager figure:

Basically in a nut shell unless you are getting an out of memory exception everything is fine and as a rule of thumb I would plan on having about 7mb of physical ram available beyond the amount consumed by the os and other applications for each instance of generator you want to run.

Also as I mentioned before we’ve made a tweak to allow a different connection to be made while the same application is using the AyaNova business object library which means an app can be loaded once then change connections sequentially to different databases and login again so that in theory only 6mb of ram in total would be needed for all your generators if it was coded to allow sequential processing of different customers databases, however you might not want to do that now that I’ve had more time to think about it:

First off there is a definite performance hit right off the bat because it takes time to connect, initialize and login which though small could add up quite a bit depending on how many different databases you process in a row, but there is a far more serious potential issue here.

Imagine that one of those databases being processed has some kind of problem with it that stops the generator or freezes it for whatever reason, now all your eggs are in one basket so if any one of those has a problem now they all have a problem because no further generation is taking place.

With the incredibly cheap cost of ram these days I personally would take the approx 6mb per database hit and run them all separately, a 1gb memory module is currently selling for as low as 9 dollars and even decent ram is 30 dollars a gb and thats enough extra ram for at least 130 generators to be running at once, however the option will be there if you want to go the other route with v5 and code up an alternative wrapper for the generation function built into the business object library.