[UD-Grid] Nieuws updates

quote: http://forum.grid.org/phpBB/viewtopic.php?t=18443

Dear Members,

Sorry for the late status, but I wanted to finish a test before posting:

New Cancer Data - I have been running some tests on an internal system and believe that I have some good news. It appears from my testing that everything is working as expected. There was a false alarm last week when I reported that we were ready to send results to our Oxford contact. Upon further investigation it was seen that there were some missing output files. This problem has been resolved internally so I am ready to grab the results from the new test job that has been running on Grid.org to see if the results are the same.

I will need to run the result aggregation script which is always run after a job is complete. An effect of running the script is that the workunits will all be marked complete and dispatching will stop. This may result in a lost workunit result or two. I will re-enable the job as soon as the result script completes so there will be minimum outage.

Lost Workunits - There have been some complaints of lost workunits with the new data. Since we have many results for each workunit, this is not a problem with the new cancer data per se. We will keep investigating this issue until we understand what is happening.
_________________
Robby Brewer
Senior Support Engineer
United Devices

I have sent the results from our small test job to our Oxford contact. We will now have to wait to see what they say. As soon as I get confirmation, I can upload all of the new data. Note that we know that there is a problem with the format of the data. I had to manually convert the data in order to run our test. I am asking our contact to provide us with the correct format so that we know the input data is valid from their point of view.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Pim. schreef op donderdag 12 januari 2006 @ 08:44:
Prima om dat dan in een centraal topic te doen ?, nu verzuipt het in DPCH's en lezen alleen de DPCH mensen het terijl er mischien leuke info voorbij komt

Dear Members,

Not a lot to mention this week:

New Cancer Data - The results from the current job have been sent to our Oxford contact. We must now wait for the results to be verified. Note that there is already one issue. Oxford is requesting an additional tag Molecule_ID be added to the input data. Hopefully this is something they will be able to deliver. As soon as I hear back, I will post.

Lost Workunits - One of our 3 servers that handle dispatch and result retrieval was not working properly. I do not see how that would cause workunits to be lost, but I guess it is possible. That server has been fixed and is working properly now. Let's keep an eye on the workunits and see if this has a positive affect on those experiencing the problem.

Dear Members,

I apologize for not posting a status on Monday, but we are having a company conference this week which is consuming most of my time. The status of the new cancer data is the topic everyone is interested in so here is a very quick status.

I have a conference call with our Oxford contact tomorrow morning to discuss the new data and the results I sent a couple of weeks ago. After the call, I hope to have much more information to share. I know there is frustration about not having all of the new data loaded for members to work on. Since Oxford is the one that will ultimately be using the results, I have had to wait for them to respond as to the validity of the results. That is out of my control.

Regardless of what Oxford has to say, I will load some more of the new data next week. If the data must be recrunched later, so be it. I was trying to minimize the amount of data that must be reworked to cut down on the complaints about wasted time later. Since there are already complaints about time wasted crunching the same workunits, I guess it does not matter.

Please understand that having the data validated by Oxford is the bottleneck here and that United Devices is ready to upload all the new data as soon as the results are confirmed.

News Dec 2005

Pancreatic Cancer project results presented at AACR-NCI-EORTC Molecular Targets Meeting Nov 2005

Our collaborators on the pancreatic cancer project: Vijay Gokhale, Haiyong Han, Daniel D Von Hoff and Laurence H Hurley working at the Arizona Cancer Center, Tucson, AZ and the Translational Genomics Research Institute, Phoenix AZ presented the results of the recent project.

"quote - Using Screensaver Lifesaver / LigandFit technology, we have identified three promising leads Mol597, Mol238 and Mol628 for urokinase inhibition. Based on the docking studies these molecules show high potential as uPA inhibitors. These molecules can be used as lead molecules for the design of better uPA inhibitors as potential anti-cancer therapeutic agents. We are synthesizing these molecules and testing them for biological activity. " PDF bestand met vrij specifieke informatie

Dear Members,

Here is the weekly status:

New Cancer Data - I had a conference call with our Oxford contact on Friday. There is an additional field they would like in the results so they have sent a sample data file containing this information. I will be uploading this later today or tomorrow. Hopefully this will produce the results they are expecting and we will be able to make all of the new data available for members later this week or next.

Rosetta - The current batch has been processed and we will be making a new batch available later this week. As always, there will be a two week period for any outstanding workunits to be credited.

Team Stats - For some reason these are missing for the 26th. The stats job will be rerun shortly to pick up this day.

Thanks to everyone for their contribution.

Dear Members,

I have just uploaded a portion of the latest data I received from Oxford. My internal tests looked good as far as the workunits being able to be processed. We will let this job run for a while before shipping the results back to Oxford for verification. Note that the previous cancer job is still running as well, so it is up to the dispatcher as to whether you get one of the new workunits or not. As soon as I see a few successful results, I will stop the previous job from dispatching so everyone will be crunching the new data.

Dear Members,

I have received confirmation from our Oxford contact that the sample result data from the latest cancer job looks good. This means that we are ready to start crunching all of the new data. The new data will be uploaded in pieces (jobs) just like we have been doing with Rosetta. When a job has completed, I will allow at least a week for any outstanding results to be uploaded by members.

This is great news for all of us. Thank you for your continued patience and contribution.

Rosetta - The current batch has been processed and we will be making a new batch available later this week. As always, there will be a two week period for any outstanding workunits to be credited.

Blue_man schreef op zaterdag 04 februari 2006 @ 09:19:
De nieuwspost van hierboven gaat over Cancer-WU's, die Rosetta's blijven normaalgezien geldig (5 Rosetta's? * Blue_man begint te rekenen )

[ Voor 22% gewijzigd door Omied op 05-02-2006 23:21 ]

Here is the weekly status:

Cancer data - As I mentioned previously, Oxford has verified our previous results and has released all of the new data. There is plenty to keep us busy for a while. They also wish to run all of the new data against the previous protein when we are done with the current protein.

Some members are experiencing aborted WUs with the new data. We are currently investigating this issue. The problem is a bit elusive since not all members are experiencing it and some members are experiencing it much more frequently than others. I have verified that there are results for each and every WU and that the total is approximately the same for all. I have also verified that there are approximately the same (small) number of errors returned for each WU.

This tells me that there is not a problem with any particular WU, but some other issue. If there was a bad WU, we would see either a significantly lower number of results or a significantly greater number of errors. This is not the case. We will keep looking into this issue until we find a solution.

These WUs appear to be processing much faster than the previous batches. I am not sure of the reason for this and will rely on Oxford to tell us if something is not right. Note that there is a bunch of data to be processed. I am not quick to modify the job configuration (adding more Ligands) since the next ones I upload may take longer.

It has been noticed that the number of hits is high for some of the WUs. Again I do not know the reason for this and will have to rely on Oxford to tell us if something is wrong.

Thank you for your contribution.

Dear Members,

We have finished processing the current Rosetta job. A new one will be uploaded tonight. Some members may have experienced a "Cannot connect" message due to this. Until the new job is active, I have reset the current one to continue to dispatch to prevent these messages.

Thank you for your contribution.

Dear Members,

I have uploaded another batch of cancer data. The last job will remain active for a few days to give members time to receive credit for any outstanding workunits. Since these new workunits are processing so quickly, I do not think we need to wait a whole week.

Thank you for your contribution.

Dear Members,

Here is the weekly status:

Cancer Data - We continue to have an issue with occassional aborts that is still under investigation. As I mentioned in the Member to Member Support thread, this issue has been identified as a problem with the Ligandfit application itself crashing. This has nothing to do with the UD servers dropping results or not giving due credit. The Ligandfit application has not changed in a very long time, prior to the last batch of cancer data that received no complaints. This would point to something with the new data that is causing Ligandfit to be unstable.

Note that we are receiving successful results from each and every workunit so there are not "bad workunits" that will always fail. Some members have stated that retrying a workunit that just aborted will result in a successful completion.

Although a bit frustrating because of lost points, know that we are getting successful results that can be returned to Oxford to assist in the search for the cure for cancer which is the ultimate objective of this project.

Dear Members,

Here is the weekly status of Grid.org.

Outage: There was a temporary outage over the weekend that caused connectivity problems with Grid.org servers. This was caused by a router failure in our datacenter and was unanticipated. The problem has since been rectified and everything should be functioning normally.

Cancer data: I have uploaded a new chunk of data and have sent the last result set to Oxford for analysis. Some members have noticed that this batch produces an awful lot of hits. I am asking if this is to be expected and if this could in any way be related to the occassional workunit aborts that some members are seeing.

Thank you for your contribution.

Dear Members,

Here is the weekly Grid.org status. Things are relatively quiet although we still have a few outstanding issues.

Cancer data - I have not heard back from our Oxford contact regarding the last batch of results I sent. This was an attempt to have them validate the many hits we are seeing on some of these workunits. It seems suspicious to me, but I am not well versed in computational chemistry and cannot speak to this. It will be up to Oxford, who provided the data, to determine if the results are valid for their research. Until then we will just keep crunching away at the new data.

There is still the issue of Ligandfit crashing. Although I do not know why, it appears that there is something about the new data that causes Ligandfit to get in a bad state. I have also mentioned this to our Oxford contact and hope that he may be able to shed some light on this issue. Perhaps the many hits or complexity of the new data is the contributing factor.

As always, thank you for your contribution.

Posted: Wed Mar 08, 2006 4:25 pm Post subject: Comment On Cancer Data

Dear Members,

I have received a response from our Oxford contact regarding some of the issues we have seen with the new cancer data.

High number of hits - It was stated that a determination could not be made at this stage as to whether the data was generating too many hits or not. After we process all of the data and send it back, they will apply a cutoff criteria in order to isolate the best hits. It may be that once the data is analyzed, they will suggest a change to the Ligandfit input parameters and a rerun of the data. Remember, research implies trial and error.

Aborts - While nothing specific was identified yet, they stated that is was possible for the data to cause Ligandfit to error out. It was mentioned that some of the datas are not discrete organic molecules which may be different that data that was processed in the past. It may be that under certain circumstances this data causes Ligandfit to abort, although we have seen that a replay of the data usually produces a good result so we are not talking about a bad WU per se. They also stated that in the future they will filter out anything that they think may cause an issue. Bottom line is this issue remains elusive and we do not have the answer yet.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Posted: Fri Mar 10, 2006 12:02 am Post subject: Server Backing Off...
--------------------------------------------------------------------------------

Dear Members,

We have received too many results again for the servers to hold. I am in the process of clearing some space now. Please be patient while I get the situation remedied. The root problem is that these workunits are processing much more quickly than the previous batch. This is causing many more results in a much shorter period of time. This equates to disk space filling up much more quickly.

It is obvious that our process needs to be changed to limit the time a job runs. I thought two weeks would be sufficient, but I think now I need to turn these around every week.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Posted: Fri Mar 10, 2006 12:03 am Post subject:
--------------------------------------------------------------------------------

I am currently uploading a new job. It has already started dispatching and I have verified a new workunit. Since we got a little backed up, it may take a while for everyone to connect and get a new workunit. I will post some info tomorrow on our new (shorter) process for turning these jobs around.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Posted: Mon Mar 13, 2006 10:01 pm Post subject: Grid.org status
--------------------------------------------------------------------------------
Dear Members,

Here is the weekly status for Grid.org.

Cancer job - As you probably noticed, we had a small outage last week due to the amount of results being returned for the new cancer jobs. These WUs are running within a few hours each probably due to the (non) complexity of the new protein. This translates to lots of disk space being used up very quickly. Due to this, I will need to roll up the results of these jobs much faster than pervious ones in order to keep below maximum disk space. Here is what I propose:

On Wednesdays I will submit a new cancer job. Once it has been activated, I will run the roll up script which should only affect the previous job. The roll up script will mark the previous job inactive as part of its processing. Once the roll up script has finished, I will reset the job to be active. This will allow outstanding results to be credited.

On Fridays, I will delete the older job that has already had the results rolled up so any outstanding workunits will not be credited. You would have had 2 days to return your results which should be plenty considering the WUs complete within hours.

There may be a small window while the results are being rolled up to where newly returned results will be rejected. This would be due to the job being temporarily marked as inactive. Unfortunately, I see no way around this. The impact should be very minimal. If you are that concerned about a lost workunit, shutdown your agent on wednesdays until I post that the job has been reactivated.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

As per our new process described on Monday, I have uploaded a new cancer job and rolled up results from the previous one. On Friday I will deactivate the previous job afterwhich no more credit will be given.

Hopefully this process will keep results at a minimum to conserve disk space, but still allow everyone to participate and receive credit for their work.

Thank you for your contribution.

Dear Members,

I have rolled up the results from the current Cancer job and submitted a new one. Credit for the previous job will be given until Friday morning. If you are running UD Mon and are concerned about getting credit for all workunits, this would be a good time to dump/refresh your cache with the new ones.

Dear Members,

There have been a few suggestions to extend the time that the previous cancer job is deleted in order to give slow machines a chance to finish crunching. We want to accomodate all members regardless of the speed of their machines so I am going to change the day to Monday. New jobs will be submitted on Wednesdays and old jobs will be deleted on Mondays.

HT does not work well with UD as LigandFit requires the FPU all the time and there is still only one to share between the 2 running UD processes (if you use the various methods mentioned here or elsewhere). HT will work if you need to share the processing between integer and FPU activities but not if both processes require integer or FPU only.

If your primary goal is crunching then turn off HT as it gives you very little benefit anyway. In some cases HT will actually slow things down because the instruction & data caches are split between both logical processors. HT only benefits if you run SMP capable apps all the time (like some graphics rendering apps etc) which you probably don't if you only have one processor.

Having a processor intensive task then effectively halving the available processor resources will only have one obvious result. You will be better off in the long run by turning off HT and overclocking the processor.

Cheers

Orbi

Tao schreef op vrijdag 17 maart 2006 @ 16:31:
Inderdaad 2 clients draaien onder aparte inlogaccounts.
Bij elke account moet je de affinity van de koe via de taskmanager op 1 en dezelfde cpu zetten.
Dus: guest1 -> alles op cpu0, bij guest2 -> alles op cpu1.
Vergeet NIET om ud.exe op compatibilitymode NT4 te zetten. (B.v. via Verkenner, rechtermuis-klik op ud.exe, dan naar eigenschappen, compat.mode, en dan NT kiezen.)
Zo draai ik ook al bijna 5 jaar onafgebroken mijn kudde

Enig probleempje is dat bij herstarten van de pc (of na uitloggen van het account) je alles opnieuw kan instellen
Mijn advies is dus: NIET MEER UITZETTEN

quote: Kewjoe

ok here goes:
WinXP allows Switching of users without closing another users programs. Its called "Fast User Switching"

As mentioned later in the thread, being on a domain disables Fast User Switching.
Next.. i have two accounts on the machine.

Account #1:
Install UD in Directory A
Run UD
Set Affinty to CPU 1

Switch Users

Account #2:
Install UD in Directory B
set Compatibility Mode to NT 4 (Service Pack 5) <--- THIS IS HOW I GOT IT TO WORK *Important*
Run UD
Set Affinity to CPU 0

From there it will work. Check Taskmanager to make sure both processors are being used.

[ Voor 46% gewijzigd door Blue_man op 26-03-2006 09:43 ]

Dear Members,

Here is the weekly status:

Cancer job - There is no new information to post at this time other than to restate that each week the previous cancer job will be deleted on Mondays instead of Wednesdays. This will give members running on slower machines a chance to return their outstanding workunits for credit.

Thank you for your contribution.

The previous cancer job has been deleted. If you have any outstanding workunits that have not yet been processed, they will not be given credit and should be dumped.

Dear Members,

Here is the weekly status update for Grid.org.

Cancer data - I have deleted the previous Cancer job so no more credit can be given for outstanding workunits. We continue to process a new job each week, which means things are going pretty well. The Ligandift aborts still continue and probably will through the entire new batch of data that Oxford provided. We will see what the behavior is when we run this data against the previous protein which will be the next task.

Thank you for your contribution.

Posted: Thu Apr 06, 2006 8:21 pm Post subject: Backing off
--------------------------------------------------------------------------------

Dear Members,

We experienced a brief outage again due to the large amount of results. I thought that rolling up once a week was sufficient, but apparently we still ran out of space. The system has been recovered now and everyone should be able to connect now.

There may be some lost workunits due to the nature of this problem. I apologize for that. A new job has been submitted so if you are running UDMon, you should dump all of your cached workunits and reload.. Credit will not be given for the previous outstanding workunits because the job had to be deleted as part of the recovery procedure.

I will be adding some additional disk monitoring tools to hopefully notify me before this issue can reoccur. Again, I thought that our weekly rollup process was sufficient. Please bear with me as we continue to tweak the process.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Blue_man schreef op vrijdag 07 april 2006 @ 19:23:
[b]Liz: zet je even in het vet dat mensen hun ud_mon slots moeten vernieuwen (bvb met fudc) en dit zo snel mogelijk

[ Voor 14% gewijzigd door Anoniem: 125039 op 07-04-2006 19:38 ]

Posted: Fri Apr 07, 2006 2:17 pm Post subject: Backing off...part two
--------------------------------------------------------------------------------
Dear Members,

There was another "backing off" issue starting last night with the cancer job only. This is completely my fault. When I ran the rollup script yesterday to merge the results of the previous job, I forgot to stop the script after the rollup was done. Since the script was still running, it started marking workunits on the new job as complete as the results came in. When all workunits had been marked complete, the job quit dispatching.

I have reset the job and everything should be working again now. There should also be no lost results/points. Again I apologize for the inconvenience and thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

All is relatively quiet. I have succeeded in putting some disk space monitors in place to hopefully prevent another disk space issue like we had last week. Thank you for your contribution.

Posted: Wed Apr 12, 2006 7:57 pm Post subject: Cancer job
--------------------------------------------------------------------------------
Dear Members,

I have rolled up results for the last cancer job and submitted another one. Results for the last cancer job will be credited until Monday. If you are running UD Mon, please dump your cache and reload with the new workunits or you may lose some credit.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Anoniem: 125039 schreef op zondag 16 april 2006 @ 18:47:
Hoe zit het nou eigenlijk duurt sowieso de tijdsverloop van een batch maar 1 week?

Dear Members,

Due to high temperatures and electrical power constraints, the state of Texas has been performing rolling blackouts in an effort to reduce load. It is possible that one of these blackouts might hit the United Devices location. If this happens Grid.org may be unavailable for a period of time. We do have UPS backup that will carry us for a while, but not indefinitely. Please be patient if Grid.org becomes unavailable. We will have it back up as soon as possible.

Dear Members,

Things are still quiet so there is not much to report. The previous cancer job has been deleted and so no credit will be given for outstanding results. As usual, I will submit a new cancer job on Wednesday.

Thank you for your contribution.

[ Voor 2% gewijzigd door Blue_man op 24-04-2006 19:20 . Reden: Rechtstreekse link ]

PostPosted: Mon May 01, 2006 2:23 pm Post subject: Grid.org status Reply with quote
Dear Members,

We continue to crunch the new cancer jobs. A new one will be uploaded on Wednesday. We also uploaded a new Rosetta job over the weekend.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

We will be performing some maintenance on one of our routers today. There may be a small outage due to this. Both the forums and the gird.org servers may be affected. The work will begin at 6 PM CST and should last 1.5 hours.

MrMaD1909 schreef op vrijdag 05 mei 2006 @ 23:02:
Als je "detect aborted WU's" aan hebt staan gooit hij ze er zelf uit als het zover is. Waarschijnlijk krijg je er dan geen punten meer voor. Het is nu zorgen dat je op donderdag een WU'tje ophaalt en die blijft crunchen tot de tweede zondag. Dan krijg je de meeste punten per WU.

quote: http://forum.grid.org/phpBB/viewtopic.php?p=555074#555074

using this prox was able to connect for WU and forum
216.55.181.230 port 3128

InSPiRE schreef op maandag 08 mei 2006 @ 16:41:
[...]

Posted: Mon May 08, 2006 5:38 pm Post subject: Unable to connect
--------------------------------------------------------------------------------
Dear Members,

After the router maintenance on Friday, some members experienced some connectivity errors. This was not limited to grid.org, but UD connectivity in general. The maintenance task that was performed dealt with Border Gateway Protocol (BGP) which allows connectivity to two separate internet providers. The idea being that if one provider goes down there will be automatic failover to the other internet provider resulting in some redundancy.

For some reason, enabling BGP on the router resulted in connectivity loss (apparently for anyone coming in through one or the other provider which is why everyone was not affected). We have not determined the reason for this yet. In the meantime, BGP has been disabled and all connectivity should have been restored for all members.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

I have rolled up the results from the previous cancer job and submitted a new one. Results for the previous job will be credited until Monday. If you are running UD Mon, you may want to dump your cache in order to process the new workunits.

Dear Members,

Here is the weekly status for Grid.org:

Cancer job: I have deleted the previous cancer job which everyone should have finished processing by now.

Team stats: These are missing for 5/12/06 for some reason. We have not had a stats problem in a while. I have requested a DBA to manually fix these when they get a chance. The stats will hopefully be up to date later today.

Thank you for your contribution

Dear Members,

Here is the weekly status.

Stats - Should all be up to date now.

Rosetta - We have submitted a new Rosetta job for those processing HPF. Results from the previous job will be credited for a while.

Cancer - Nothing new to report. We are still seeing workunit aborts, but the root cause has not been identified yet.

Thank you for your contribution

Dear Members,

I have rolled up the results for the previous cancer job and submitted a new one. As usual, outstanding results will be credited until Monday.

Dear Members,

The latest Rosetta job completed over the weekend. This caused some members to experience the "cannot connect..." messages. I have reset the job so that outstanding results can still be credited and verified that I could download a workunit. Everyone should be able to connect now.

Thank you for your contribution.

Dear Members,

Here is the weekly status:

Cancer job - Oxford would like us to finish the current batch of data by the end of June. This means I will have to start submitting multiple jobs and rolling them up a bit sooner than we have been doing. Since these workunits process so quickly even on slow machines, it should not affect anyones ability to return results for credit even if I cut the processing time by a few days. I will make posts as I submit/rollup jobs.
Thank you for your contribution.

Dear Members,

I have rolled up the previous cancer job and submitted two more. You will find that these workunits have twice as many Ligands as the previous ones. I verified that I could download one and it appears to be processing fine. Depending on how many results we receive, I may roll these up sooner than next Wednesday.

Dear Members,

As I mentioned last week, I am having to accelerate the cancer jobs so that I can have results to Oxford by the end of the month. Today I rolled up the results from the previous two jobs and submitted two more. Results for the previous job will be credited until Wednesday, which should give everyone a chance to return their results.

Thank you for your contribution.

[ Voor 74% gewijzigd door Anoniem: 156750 op 13-06-2006 15:37 ]

Dear Members,

I have rolled up the results from the last two jobs. Note that unlike the other jobs that had 1 or 2 bad WUs, these last jobs had several 40 - 50. This would account for the higher rates of aborts seen in this last batch. I have submitted three more jobs. Results for the previous jobs will be credited until at least Wednesday.

Dear Members,

Here is the (late) update for Grid.org. I apologize for my late post, but there have been some support issues that have consumed much of my time recently.

Cancer job: I will be submitting a new job today that will actually consist of workunits that we have already processed. The reason for this is that I must submit at least something before I do my result rollup or else members will get the "cannot connect..." message. I will then rollup the results of the current cancer jobs. This will be the final results for all data that Oxford has provided for us to crunch. They will analyze it and hopefully will not require a respin.

I will then look at submitting this same cancer data set against the previous protein that was used. It may take a couple of days to get the job correct so please be patient.

Stats: The team stats did not update the other day so I had to manually rerun the task yesterday. This would affect the stats task that was supposed to run last night so I am rerunning again currently. Everything should be caught up by tomorrow.

Connectivity: Some members have reported some connectivity problems to the grid.org portal - members services, forums, etc. Other members are able to connect with no problem. We have not been able to identify what may be happening here. I have not been able to reproduce this issue and there have been no changes to the UD hardware/network. We are still looking into this and if anyone has any additional info, please let me know.

Thank you for your contribution.

Dear Members,

I have rolled up the results from the last batches of data, which completes that batch of data against the current protein. I was also asked by Oxford to run the same data against the previous protein. Until I hear otherwise, that is what we are going to do. Today I submitted a new job that runs the first batch of the latest data against the previous protein we ran against late last year.

This job seems to be taking much longer than the ones we have been running lately. Therefore, I have reduced the ligand count back down to 600. Some members may have received a slow 1200 ligand workunit. If so, please dump that workunit because I have deleted the job and no credit will be given for results.

Finally, it will take a while for one of these workunits to complete. It is possible that there is some sort of problem with these workunits and that they may fail in the final stages of execution. I do not expect this, but I did not get a chance to test this on another system first. If there is some sort of a problem, we will deal with it quickly.

It will be interesting to see if we get any aborts like we did against the previous protein and what that rate is.

Thank you for your contribution.

Posted: Mon Jul 03, 2006 3:54 pm Post subject: Grid.org / Cancer Job Status
--------------------------------------------------------------------------------
Dear Members,

Here is the weekly status.

Cancer Job - Obviously, the WUs are taking too long to process. I tried to upload a new job as soon as possible after rolling up the last batch because many members were talking about leaving. Therefore I did not test how long the new WUs would take, but just that they would indeed process. These long running WUs have caused a harsher response apparently than not having anything to crunch. I guess I can't win either way.

I have suspended the 600 ligand job so that no more WUs will be sent out. I have uploaded a new job with a total of 100 ligands. I have not tested how long these will now take. If it is still takes too long, we will reduce the number of ligands further. There is no built in timeout in the previous 600 ligand WUs, so if you want to continue crunching on them you can. I will wait a reasonable amount of time before deleting that job so that credit may be given. If you do not want to continue, delete the WU and you will get one of the 100 ligand ones.

Rosetta Job - I think I mentioned this before, but some members have asked me recently. The HPF project that we were working on is apparently done. There are no more new batches of data available to us to work on. I have heard nothing about Grid.org participating in HPF2 if there is such a thing. The WUs that are currently active have all been previously processed so the work is redundant. I am not going to delete it though until I am sure that there will be no more HPF projects for Grid.org to work on.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Posted: Mon Jul 10, 2006 3:20 pm Post subject: Grid.org Status
--------------------------------------------------------------------------------
Dear Members,

Here is te weekly Grid.org status.

Cancer - After a few false starts, I think we have a good Ligand number at 100. These take much longer than the previous ones, but it seems that even a slow machine should be able to crunch a single WU in one week. Unless I hear otherwise, we will stick with 100 ligand WUs.

It also appears that the aborts have gone away. Since nothing has changed other than the protein we are working on (as was the case when they started happening), I think this proves that there was something in the last protein that the Ligandfit application had an occassional problem with and not an issue with UD hardware. Since Oxford delivered the protein, they would have to be the ones to figure out what is different in the two proteins.

Regarding the 600 ligand job, I will let that one spin for another week or two to give everyone a chance to return their results. Otherwise, we will go back to the one week cycle we were on previously with the 100 ligand WUs.

Rosetta - This project is apparently complete. Since I do not know if we will be doing any other Rosetta projects, I am going to leave this running until I hear something official. This means currently any Rosetta work is for points only. The actual work will be ignored. If I turn the job off, I am sure that many machines will get the flashing explanation point and cause an uproar. Active members who read the boards can make a decision whether they want to set their profile to the cancer job only or not. Maybe some members think the Rosetta screensaver is cool.

Connection issues - We have had random connection issues recently. Restarting the load balancer last week appeared to have fixed the issue, but I noticed a few complaints today. I do not know if this is the same issue or something different. There is a request into IT to check the load balancer again to see if it has some sort of chronic problem. In the meantime, I can only suggest to retry until you get the page you want. There is a 50/50 chance you will get directed correctly if the load balancer is acting up. Luckily these are not agent connection issue, but limited to the member web pages.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Posted: Mon Jul 17, 2006 6:38 pm Post subject: Grid.org/Cancer Job Status
--------------------------------------------------------------------------------
Dear Members,

Here is the weekly status.

Cancer Job - I received several PMs asking to extend the result credit time on the previous jobs since they were taking so long. Even the 100 Ligand WUs were taking a while for some members. I have submitted a new 50 Ligand WU job to try to get the processing time down. We will see how this works out.

We want all members to be able to participate regardless of the power of their machines. I will leave the current jobs active for at least another week to give time for results to come in. Note that the jobs must be set inactive as always while I roll up the results. Hopefully you will not have the bad luck of trying to return one of these large WU results while the job is inactive.

Please post in a couple days to let me know how the 50 Ligand WUs are working out.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

No they won't overlap. UD Cancer project is fitting prospective molecules (ligands) against target cancer proteins. The WCG cancer project is scanning tissue samples for cancer biomarkers (so that doctors will be able to identify cancer in early stages and select the proper treatment based on certain biomarkers).

btw, the memory requirements for WCG's Help Defeat Cancer project are 750MB of RAM. also, I'm reading that there are a relatively small number of workunits - apparently the Cancer Institute of New Jersey has to do some preliminary processing of the TMAs before they can be sent out as workunits (and they haven't figured out a way to make this preliminary step distributed).

Dear members,

Here is the weekly status.

Cancer job - Nothing new to report. We continue to crunch through the data. I uploaded a new batch today so there should be new workunits available.

Posted: Mon Oct 09, 2006 4:25 pm Post subject: Grid.org status
--------------------------------------------------------------------------------
Dear Members,

Here is the weekly status.

Cancer Job - No change. I did upload some new WUs late last week. Everything appears to be running smoothly.

Stats - We still have an issue with a couple of days of team stats - 10/5 and 10/8. I am going to manually kick off the rollup script so hopefully both will be fixed by tomorrow. As far as exactly why this occurs, I am not a mssqlserver expert (by far). Occassionally the stored procedure that does the work times out and does not complete. Nothing is lost and it is a simple matter to kick off the procedure again by hand so members should not panic about this.

Thank you for your contribution.
_________________
Robby Brewer
Senior Support Engineer
United Devices

PostPosted: Mon Oct 23, 2006 4:28 pm Post subject: Grid.org status Reply with quote
Dear Members,

Here is the weekly status.

Stats - We have an issue with stats currently. For some reason, the stats job keeps failing. We have not identified the problem yet. Nothing should be lost as far as points. We just need to get the job to run correctly to tally up all of the stats. I am not sure when this will be fixed so please be patient.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

Here is the weekly status.

Stats - We have an issue with stats currently. For some reason, the stats job keeps failing. We have not identified the problem yet. Nothing should be lost as far as points. We just need to get the job to run correctly to tally up all of the stats. I am not sure when this will be fixed so please be patient.

Dear Members,

Here is the weekly status.

Stats - The stats continue to be a bit erratic. We have not made any changes recently that would cause this behavior. They appear to eventually catch up, but are often a day or two behind. I am starting to suspect that this is because we have crossed some critical threshold of stats in the database (i.e. - DB has grown very big over the years). I think the stats job is not always able to finish running in the alloted time frame. We will continue to look into this, but know that we may need to continue to manually complete the stats rollup for a while. I will be manually running the team stats shortly.

Thank you for your contribution.

Dear Members,

Here is the weekly status.

Stats - We are missing team stats for 11/8. I am manually rolling those up now so everything should be caught up in a few hours.

Thank you for your contribution.

Blue_man schreef op maandag 13 november 2006 @ 20:10:
[...]

Dear Members,

Here is the weekly status. I was out of the office on vacation last week which is why I did not post last week's status.

Stats - I see that some members are having issues accessing the stats. Obviously it should not take 10 minutes to bring up the page. It appears that there may be an issue with one of the load balanced servers. I will have IT look into this as soon as possible.

Cancer - I will be uploading some fresh workunits today.

Thank you for your contribution.

Dear Members,

Here is the weekly status:

Stats - Team stats have been running a bit behind, but there has not been any manual intervention needed recently.

Forum/Member Services - Our IT department recently reset all servers. I personally have not experienced the slow web page times since. Hopefully this issue is resolved for all members.

Thank you for your contribution.

Dear Members,

Here is quick update on Grid.org:

The stats rollup had another glitch which has now been resolved. All stats are now up to date. We have still not identified the root cause of this sporadic issue, but are still looking into it. I will keep my troubleshooting progress to myself until we are sure we have identified the root cause since it just seems to generate more complaints from the member community.

There has been concern from many members about Grid.org in general as far as the status of projects, new work, software upgrades, etc. Please expect a posting next week from a UD executive regarding the future of Grid.org. (Before any conspiracy theories get started, we are not announcing that we are shutting down Grid.org).

As always, thank you for your continued contribution.

[ Voor 1% gewijzigd door Blue_man op 09-12-2006 09:47 . Reden: Mijn bedoeling iets duidelijker gemaakt. ]

Blue_man schreef op vrijdag 08 december 2006 @ 20:45:

Greetings and Happy Holidays!

First of all, on behalf of United Devices I’d like to thank each and every one of you for your steadfast support of this unique, invaluable resource. For the last several years, Grid.org has played an important role in some tremendously important research and it is the largest and most respected volunteer-based grid of its kind in existence. None of that would be possible without all of you!

We have some exciting information to convey concerning Grid.org status and plans – but before I do that, please allow me to express our sincere regret for the frustrations that some of you have been experiencing lately. During the past year we have had technical difficulties with Grid.org that have contributed to your frustrations, but we have changes planned that should significantly improve your experience.

These are some of the plan highlights we have for Grid.org in the coming months:

We will migrate the Grid.org software to the latest version. Planning for this is well underway, and the migration will begin in the first quarter of 2007! In the first quarter (Jan-Mar) of 2007, we’ll begin a Beta Program (see below) in preparation for the final upgrade to the new release. We will perform the entire upgrade to the new version by end of Q2 of 2007.

The following new features will be introduced with the latest version.

- Support for multi-processor and multi-core machines
- Support for cache ahead so that users can request multiple work units
- Enhanced non-intrusiveness options
- The possibility for members to generate their own tools, GUIs, and reports using a new Grid MP Agent API
- New look and feel for the Grid MP Agent GUI
- Improved support for Microsoft XP operating system – enhances support for multi-users and allows fast user switching
- Support for Microsoft Vista operating system (both 32-bit and 64-bit)
- Support for Microsoft Internet Explorer 7
- Overall performance improvements to the grid
- General defect repairs

Right now our outreach committee is actively working on reviewing and selecting new projects to run on the updated Grid.org software. The initial conversations we’ve been having look very promising and we should be ready to make an announcement about the selected project by end of Q1 2007, to launch in the May-June timeframe. Your active co-operation in the Beta program will be key toward expediting this process!

As mentioned above, we will implement a Beta program and ask for volunteers to help with Beta testing of the new system. This program will initially be implemented with a target of 100 volunteers as the first phase of the upgrade to the new version of Grid.org. The Beta program will run for several weeks and is intended to identify any deployment issues that might be lingering in the new version so we can resolve them prior to the upgrade of the entire member community. We will provide you with additional details and initiate a call for Beta volunteers shortly before we begin the program.

Thank you once again for your continued support of Grid.org. We’re looking forward to keeping you informed on status of the initiatives outlined above – visit this site regularly for more information.


Sincerely,

Mary Bass
Directory of Marketing
United Devices, Inc.
Austin, Texas 78759

Dear Members,

We are aware of the stats issue, but will not be able to do anything about it for a couple of hours. As part of our routine maintenance, we are backing up the stats database. This process used to occur late in the evening, which is also when the stats rollup job runs. Since we have been having stats rollup problems, we decided to not run the database backup at the same time that the rollup was running, thinking that this may be contributing to the job periodically failing to complete.

Until we do a bit more investigation, we are not sure if the backup is causing the "stats still running" issue or if this was coincidence. If the backup turns out to be the cause, we will move the backups back to late night even if it causes stats to not complete occassionally. Please be patient while we work through this. Database backups are essential so that we do not have the catastrophic data loss like we had a year ago.

I will also post this in Member News so that everyone is aware of what is going on.
_________________
Robby Brewer
Senior Support Engineer
United Devices

Dear Members,

United Devices is closed Dec 25, 2006 - Jan 1, 2007 for the holidays. I am aware of the current issue and will try to contact our IT department, but the issue may not be resolved until after the holidays. I apologize for this inconvenience.

Dear Members,

Here is the weekly status update.

Member Services - The issue we had over the holidays was due to a problem with a RAID controller on the member services server. Maintenance was performed on the RAID controller and the system has been brought back online. Although having no access to member services was an inconvenience, grid processing itself was not affected so there should be no lost work.

Team stats - Due to the hardware failure, team stats are a bit behind. I will begin manual rollup of the stats, but please be aware that this may take until tomorrow to finish. Stats must be rolled up one day at a time and it takes a couple of hours for each day to complete.

I hope everyone had a safe and happy New Year. As always, thank you for your continued contribution.

Dear Members,

Here is the weekly status.

Stats - The stats have all been manually run and are up to date. As I mentioned previously, we have identified the problem as a conflict between our DB backups and the stats rollup job. We must take backups so this issue will persist until the migration. I will manually fix the stats whenever we have this issue and nothing will be lost.

Forums - I have received several complaints about posts in the forums. There is a lot of bickering by some members that is not productive and serves no purpose. The purpose of the Member to Member Support board (and others) is to help members that are having problems and disseminate useful information related to the Grid.org project. Please ask yourself before you post if you are contributing useful information that is helpful to other members or are you are posting something that is purposely antagonistic in order to prove a point. If your post does not contribute something useful, please refrain from posting or post to UDDer Nonsense where it may be more appreciated.

Thank you for your contribution.

Blue_man schreef op maandag 15 januari 2007 @ 20:13:
[...]

Het tweede deel van het bericht is minder interessant voor ons, maar het eerste deel beloofd niet veel goeds! Ik hou het in het ook wanneer er beta-testers gevraagd worden en hoop een aantal van jullie te mogen verwelkomen in het beta-testers gezinnetje.

Witlof schreef op dinsdag 23 januari 2007 @ 16:32:
Wat ik me eigenlijk af vroeg, aangezien het in de hitparades niet beantwoord is, is of er enig nieuws is mbt een nieuwe client die wel meerdere cpu's/cores ondersteund. Heeft iemand daar wat meer info over? Blue-man bijvoorbeeld?

The following new features will be introduced with the latest version.

- Support for multi-processor and multi-core machines
- Support for cache ahead so that users can request multiple work units
- Enhanced non-intrusiveness options
- The possibility for members to generate their own tools, GUIs, and reports using a new Grid MP Agent API
- New look and feel for the Grid MP Agent GUI
- Improved support for Microsoft XP operating system – enhances support for multi-users and allows fast user switching
- Support for Microsoft Vista operating system (both 32-bit and 64-bit)
- Support for Microsoft Internet Explorer 7
- Overall performance improvements to the grid
- General defect repairs

Dear Members,

Here is the weekly status.

Connectivity - some members had problems with connectivity on Friday. I was able to connect and download a fresh WU so it does not appear that the system was down in any way. I suspect that there were some sporadic outages across the internet that caused some members to not be able to connect. Everything seems to be working fine now though.

Thank you for your contribution.