- Jan 10, 2011
-
-
Iustin Pop authored
This small patch actually reads the allocation policy from the IAllocator message. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Currently, we list the entire object in error messages. But for large objects (e.g. an IAllocator message), this makes the output unreadable, as the elements are containers themselves. To simplify the reporting, we only list the keys, as this is more relevent than the values. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Currently, fromObj/maybeFromObj take first the key, and then the object. This is suboptimal, as this form is not easy to use with partial functional application. To make it easier to switch between tryFromObj, fromObj and maybeFromObj, we unify the latter two functions to the same order (object and then key) like the first one. The code churn in the other modules is due to this, but the main change in this patch is in Utils.hs. Furthermore, since we change anyway the calls, we do replace fromObj with tryFromObj in IAllocator.hs so that we get better error messages. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
- Jan 07, 2011
-
-
Iustin Pop authored
This patch does the necessary changes to make the new tryMGEvac work correctly: each instance remains inside its primary node's group when it is evacuated. This is done by splitting up the to-be-evacuated instance list per group, together with the group node and instance list, and then running the old tryEvac for each group. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This makes the code more readable, which will help with the multi-group evacuation. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Currently, the Rapi.getUrl function returns the body without evaluating it, and the other functions (loadData, parseData) do the same. In effect, the top-level structure returned from loadData can be a thunk which depends on the curl operation, thus keeping the curl resources alive even after return from the Rapi module. While this is not a problem in single-threaded programs, it can become an issue with the threaded runtime. So it's better to fully evaluate the HTTP body before returning from the getUrl function, so that at least the curl resources are released. Either "return $! body" or "(code, !body) <- …" works (i.e. avoids the segfaults in my tests), but I think it's cleaner to force full evaluation of the body when returning from the curlGetString function. For this, we also enable the BangPatterns language extension. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
A small cleanup, this just moves the options to a separate list to avoid instantiation at every call. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
There is a long-standing bug in hscan, due to the serialization workflow. Basically, hscan loads data from the remote cluster(s), processes it via Loader.mergeData and Loader.checkData, then serializes it. The Loader function will update the node memory values as if the down instances were running, and then hscan saves the _modified_ values. When another program (e.g. hbal) loads the text files, it will re-apply the Loader functions, in effect subtracting the instance memory twice, which is course offers a wrong cluster view. To fix this bug, we change a bit the internal workflow in hscan, so that we serialize the unmodified cluster data, and only use the modified one for the display of cluster state (and verification that yes, we can load the data). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This is similar to iMem. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This exports the iMem function as a standalone function, instead of being hardcoded in showField. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This patch adds a new top-level relocation function that restricts the relocation to the instance's group, and switches hail to it. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
- Dec 30, 2010
-
-
Iustin Pop authored
Container.fromAssocList is just a re-export of IntMap.fromList; it makes sense to remove it and simply export the original name, as it needs just a bit of renaming in the rest of the code. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This small patch changes the balancing computation to work in parallel, if possible. While the normal linking is against the single-threaded runtime, if the code is linked against the multi-threaded one, the balancing will get a significant speedup (80% efficiency at 4 cores, 60% at 12 cores, and with GC tweaks it can reach 70%+). On the single-threaded runtime, due to the fact that we only use the weak head normal form, it doesn't introduce any extra penalties, neither in space nor in CPU time (or if there are, they are too small to detect easily). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Currently, the allocation routines (iterateAlloc and tieredAlloc) return only the final state of the cluster and the list of allocated instances. For better visibility in how the cluster resources change, we compute and return a list of cluster resources at each step, which should be a good indicator of the cluster state. The cost of the computation for the new resource when not used is undetectable (< 1%). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This is a cheap way to get back compatibility with Ganeti 2.3 (and lower) in the RAPI backend. It is however not very safe (the /2/groups resource could fail due to other reasons), so it is added only temporarily. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This allows a cluster structure to be easily serialized via "read"; together with the already existing instances of Show, this gives a poor man's serialization/deserialization implementation. The patch also exports the compDetailedCV function from Cluster.hs, so that it can be used by other modules too. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Just another cleanup of duplicate code. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This again abstracts a bit the instance listing. Due to the fact that I don't want to import Cluster.hs in CLI.hs, we pass the already generated output. It also moves the instance display to stderr. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Since this bit of code (including the “when (isJust …)” is used in multiple places, let's abstract it in a function that is used consistently. One (bad?) side-effect is that all node lists are done to stderr, including the ones from hbal where it was previously done to stdout (is that fine?). Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This functionality was replicated in multiple places (hbal & hspace), so we abstract it for better clarity. Additionally, in hbal we now save the state both before and after balancing. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
We change processData to just load the data, and return a simpler type. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This patch converts the backends and mergeData to the new ClusterData type. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This will be used to hold all the disparate uses of the cluster data: we have either tuples with these four elements, or functions taking these four arguments, etc. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This patch moves the allocation policy from hardcoded to be read from the given specification, and extends the error message for invalid specifications. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
This patch changes the behaviour of the --simulation option to be an incremental option, where each new use defines a new node group. This allows simulation of more complex clusters. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
- Dec 23, 2010
-
-
Iustin Pop authored
* stable-0.2: Move man files to man/ subdirectory Conflicts (all removed): man/hail.1 man/hbal.1 man/hscan.1 man/hspace.1
-
Iustin Pop authored
This is just change on the 0.2 branch to synchronize with the master branch. It allows automated builds to work better across the two versions. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Adeodato Simo <dato@google.com>
-
Iustin Pop authored
* devel-0.2: Update NEWS file for 0.2.8 release hbal: return meaningful exit code for job failures Change the balancing function
-
Iustin Pop authored
Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Adeodato Simo <dato@google.com>
-
Iustin Pop authored
Currently, LUXI job failures only display a warning message, while still returning a success exit code. We change hbal to return true/false from within execJobSet/runJobSet, and add a wrapper for simpler code. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Adeodato Simo <dato@google.com>
-
Iustin Pop authored
Currently the balancing function is a modified version of the standard deviation (stddev divided by list length), due to historical reasons. While this works fine for small clusters, for big clusters it makes the balancing effect too "weak", and in some cases it refuses to balance correctly some clusters. It also makes the balancing behaviour dependant on the cluster size, which is a big no-no. Therefore we revert to the normal version of standard deviation, and we also rename the function to reflect what it does. The new version correctly balances some corner cases that the previous version didn't, and passes the current balancing unittests. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Adeodato Simo <dato@google.com>
-
Iustin Pop authored
This splits out a bit of code from hspace.hs and moves it into its own function in Cluster.hs. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
- Dec 20, 2010
-
-
Iustin Pop authored
This patch changes the allocate mode to respect the alloc_policy for groups. It does this by changing the sort key from simply the solution score, to a tuple with two elements: the alloc policy (which is now an Ord instance) and the solution score. Also, the unallocable groups are filtered out in the filterMGResults phase. The patch also slightly enhances the informational message by including the policy in the group information, to help debugging. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-
Iustin Pop authored
Currently, it's not easy to generate “fake” IAllocator request files for hail. As such, testing on simulated clusters is hard to do. To workaround this, we change hail to also take the ‘-t’ and ‘--simulate’ options, so that we can override the cluster data read from the request. Note that this will not change the request itself (so for example an evacuation will need to make sure uses the correct node names), but it's a step forward in testing hail. The other tools already can use text files which allow for better flexibility. Signed-off-by:
Iustin Pop <iustin@google.com> Reviewed-by:
Balazs Lecz <leczb@google.com>
-