This is a high level description of how to execute parallel tasks in Morpheus Cloud Management Platform.
For more info on Morpheus and the concepts mentioned here:
https://docs.morpheusdata.com/en/latest/
- This method should work for any combination of task and/or workflows calling each other to run in parallel. Workflows can be Operational or Provisioning.
- The problem is that Morpheus workflows give no method to execute a task in the background, or to execute 2 tasks in parallel.
- This may be desirable in certain circumstances. This technique came from a desire to to actively monitor and log the steps of the greater Provision and Post Provisioning workflows beyond the builtin logging facilities of Morpheus.
!!!This is only meant to explain the technique, not to produce working code!!!
Create the second task
We start by creating the second task.
- This will be the task we want to break off from the main task or workflow and execute in parallel with the initiating task or workflow.
Code
##Name: ParallelTask
import requests
import json
morphURL = morpheus['morpheus']['applianceUrl']
apiToken = morpheus['morpheus']['apiAccessToken']
headers = {"authorization": "Bearer " + apiToken, "accept": "application/json", "content-type": "application/json"}
print(f'MorphuesURL: {morphURL}')
for i in range(1,20):
time.sleep(5)
print(f'Doing Stuff: {i}')
time.sleep(5)
Explanation
This is super simple code that could be used as a Morpheus python task. Hopefully your task will have much more important things to do.
1: Note the name at the top of the code. While not important to the code, we will assume this is the name of the task in Morpheus.
2-8: Standard imports, pulling info from the Morpheus JSON and building a header for later use.
22-14: A simple loop to simulate the task doing some work.
Create first task
Write the first task. This will be the task the runs initially and will call the second task.
Code
import requests
import json
from morpheuscypher import Cypher
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
morphURL = morpheus['morpheus']['applianceUrl']
morphToken = morpheus['morpheus']['apiAccessToken']
headers = {"authorization": "Bearer " + morphToken, "accept": "application/json",
"content-type": "application/json"}
taskName = "ParallelTask"
def findwfID(taskName):
wfURL = f"{morphURL}/api/task-sets?name={taskName}"
results = requests.get(f'{wfURL}',
verify=False, headers=headers)
wfJSON = json.loads(results.content.decode('utf-8'))
try:
wfID = wfJSON['taskSets'][0]['id']
except:
print("Workflow Not Found")
exit(1)
return wfID
def execWF(wfID, instanceID):
execWFURL = f"{morphURL}/api/task-sets/{wfID}/execute"
payload = {"job": {
"customOptions": {"OPTION1": ''
}
}}
payload['job']['customOptions']['OPTION1'] = "ValueONE"
results = requests.put(execWFURL, verify=False, json=payload, headers=headers)
wfJSON = json.loads(results.content.decode('utf-8'))
if wfJSON['success'] != True:
print("!!!!Secondary WorkFLow failed to start!!!!")
exit(1)
wfID = findwfID(workflowName)
execWF(wfID)
Explanation
Again, this is a fairly simple and common outline for a Morpheus task.
1-10: Standard imports, pulling info from the Morpheus JSON and building a header for later use.
13: Sets the task name that we are going to call to run in parallel.
14-28: The function called findwfID simply finds the workflow or task ID give then task name and returns the value.
30-43: The function exeWF calls the Morpheus api to execute the second workflow.
45-46: These lines call the other two functions.
Once the "if" statement on line 41 is evaluated the task will finish with a return code of 0, or 1. If 0, then the greater Morpheus workflow will continue to it's next task while the parallel task continues to run in the background.
If the task returns 1, then the Morpheus workflow will fail as normal.
Conclusion/Additional Thoughts.
This technique could be used in number of ways and in a number of situations.
- Provision tasks that take a long time could be executed in parallel while other tasks are executing.
- Executing a handful of tasks at one time could allow a workflow with a lot of tasks to execute in a fraction of the time.
- One workflow or task could execute sub-tasks on multiple machines at the same time.
Things to look out for.
- In the case of Provisioning workflows, if the main workflow branch reboots while the parallel task is running, issues will no doubt arise.
- Keeping track of successes and failures of child tasks, especially if there are multiple child tasks, would need to be accomplished.
Leave a Reply