Charm++ parallel programming system

[Jozsef Bakosi <jbakosi AT lanl.gov>]

Hi folks,

I feel like I have a serious gap in my understanding regarding SDAG triggers.
Consider the following SDAG and code:

SDAG:

A -- C
    /
B -/

B -- D

In words:
* Task C can only start when both A and B have completed. 
* Task D can only start when B has completed.

First implementation:

.ci:

entry void wait4C {
  when A_complete(), B_complete() serial { C(); } };

entry void wait4D {
  when B_complete() serial { D(); } };

.C:

void f() {
  ...
  A_complete();
}

void g() {
  ...
  B_complete();
  B_complete();
}

Second implementation:

.ci:

entry void wait4C {
  when A_complete(), B1_complete() serial { C(); } };

entry void wait4D {
  when B2_complete() serial { D(); } };

.C:

void f() {
  ...
  A_complete();
}

void g() {
  ...
  B1_complete();
  B2_complete();
}

Thus, the only difference is that in the second implementation the SDAG 
triggers
have unique names B1_complete and B2_complete.

My understanding, based on
https://lists.cs.illinois.edu/lists/arc/charm/2016-09/msg00002.html, has been 
so
that if I have two SDAG triggers (or messages) that will be consumed at two
different when clauses, I have to put it in the queue twice, hence the code in
g(). I also assumed that giving unique names to B1 and B2 in this case should
not change the logic, since they will just be consumed once at two different
when statements.

I tried this at one part of my code and it worked, or at least I thought it 
did.
However, I was just able to produce an example, doing the same at another part
of my code, where the first one works while the second one hangs, which tells 
me
that I'm missing something.

Can someone please explain what I might be doing wrong? Or rather how I am
thinking about this the wrong way? Also, can this behavior be influenced by
other logic in the code?

In case you want to reproduce the problem, please see
https://github.com/quinoacomputing/quinoa/pull/146#discussion_r124055283.

Thanks,
Jozsef