Retrieve Rok Task Logs

This guide contains instructions to retrieve the logs of a Rok API task.

Note

Rok uses tasks to snapshot and restore volumes, as well as other Kubernetes resources such as Notebooks and StatefulSets.

What You’ll Need

Procedure

Select one of the following options, based on your desired environment.

  1. Select Snapshots from the left-side menu.

    ../../_images/menu-snapshots.png
  2. Select the namespace containing the task.

    ../../_images/rok-namespace-selector.png
  3. Go inside the bucket that contains the task.

  4. Click on the Tasks tab to view the list of tasks in the bucket.

    ../../_images/rok-tab-tasks.png
  5. Find the task for which you need to retrieve logs.

    Note

    Currently, in the Rok UI, you cannot search for the task ID. This is something we will tackle in a next release. You can use the

    • Action column to find tasks related to a snapshotting or presentation action.
    • Status column to find tasks that have succeeded or failed.
    • Scheduled column to find the exact day and time that this task was created.
  6. Verify that the ID of the task is the one you are looking for by clicking on the information icon (i) on the right side of the list.

    Note

    The IDs of subtasks, for example 273d828cce56464dacf44b8bd2c8d87c.8f64e1f2c5 contain the ID of their parent, for example 273d828cce56464dacf44b8bd2c8d87c, as a prefix. This means that if you find a task whose ID is a prefix of the ID you are looking for, then the task you are looking for is a subtask of this task.

    ../../_images/task-list.png
  7. Click on the task to view its details.

  8. Scroll down to the Task logs section to view its logs.

  9. If the task has failed, look for log entries with ERROR severity, since they contain useful information about the error.

    ../../_images/task-logs.png
  10. If you see errors mentioning that a subtask of the task you are currently inspecting has failed, then inspect the logs of the subtask as well, since they are likely to contain useful information about the cause of the error. To do so, follow these steps:

    1. Click on the Tasks tab to return to the list of tasks.

    2. Find the task you were inspecting previously.

    3. Click on the arrow on the left side of the task to expand its subtasks. Some subtasks can have subtasks of their own, so you may have to repeat this step more than once.

      ../../_images/task-list-error.png
    4. Locate any subtasks that failed and click on them to view their logs.

    5. Repeat the above steps for all subtasks that have failed.

  1. Switch to your management environment and exec into the Rok master pod:

    root@rok-tools:~# kubectl exec -ti -n rok service/rok -- /bin/bash
  2. Specify the namespace the task belongs to:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export ROK_ACCOUNT="<NAMESPACE>"

    Replace <NAMESPACE> with the namespace, for example:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export ROK_ACCOUNT="kubeflow-user"
  3. Specify the bucket the task belongs to:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export BUCKET="<BUCKET>"

    Replace <BUCKET> with the bucket name, for example:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export BUCKET="Snapshots"
  4. Specify the ID of the task:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export TASK_ID="<ID>"

    Replace <ID> with the task ID, for example:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# export TASK_ID="5ce75fb71fdc43478d4c8909e4dbe6bf"
  5. Retrieve the task, its subtasks, and their logs:

    root@rok-wmczj|rok-aks-workers-28187021-vmss000000|rok.rok.svc.cluster.local:/# rok task-show ${BUCKET?} ${TASK_ID?} --detail --logs --subtasks Registratin Policy Task Account kubeflow-user ... Time Level Message 2021-09-01T14:10:33.681198+00:00 INFO Starting task 2021-09-01T14:10:33.684596+00:00 INFO Running registration policy `e233a7c9-e2db-4061-ae3c-5a704cccc6c8' ...
  1. Create a notebook server and connect to the notebook using the Kubeflow UI.

  2. Start a new terminal inside the notebook.

  3. Create a new Python file and name it tasklogs.py:

    jovyan@mynotebook:~$ touch tasklogs.py
  4. Copy and paste the following code inside tasklogs.py:

    tasklogs.py
    1# Copyright © 2022 Arrikto Inc. All Rights Reserved.
    2
    3"""Retrieve the logs of a Rok API task."""
    4
    5from rok_gw_client import RokClient
    6
    7# ACCOUNT = "kubeflow-user"
    8BUCKET = "mybucket"
    9TASK_ID = "mytaskid"
    10
    11# Initialize the Rok client. Don't specify the `account` keyword argument,
    12# so the client uses the namespace of the current pod by default.
    13client = RokClient()
    14
    15# Retrieve the task and its subtasks
    16task = client.task_get(BUCKET, TASK_ID, subtasks=True)
    17
    18
    19# Print the logs of the task and its subtasks
    20def print_logs(task):
    21 print("Logs for task '%s':" % task["id"])
    22 for line in task["log"]:
    23 print(line["time"], line["level"], line["msg"])
    24 for subtask in task["subtasks"]:
    25 print_logs(subtask)
    26
    27
    28print_logs(task)

    Alternatively, download this file and upload it to your notebook.

  5. Update the BUCKET and TASK_ID variables inside the script with the bucket name and UUID of your task, for example:

    tasklogs-updated.py
    1# Copyright © 2022 Arrikto Inc. All Rights Reserved.
    2
    3"""Retrieve the logs of a Rok API task."""
    4-4
    4
    5from rok_gw_client import RokClient
    6
    7# ACCOUNT = "kubeflow-user"
    8-BUCKET = "mybucket"
    9-TASK_ID = "mytaskid"
    10+BUCKET = "Snapshots"
    11+TASK_ID = "5ce75fb71fdc43478d4c8909e4dbe6bf"
    12
    13# Initialize the Rok client. Don't specify the `account` keyword argument,
    14# so the client uses the namespace of the current pod by default.
    15-27
    15client = RokClient()
    16
    17# Retrieve the task and its subtasks
    18task = client.task_get(BUCKET, TASK_ID, subtasks=True)
    19
    20
    21# Print the logs of the task and its subtasks
    22def print_logs(task):
    23 print("Logs for task '%s':" % task["id"])
    24 for line in task["log"]:
    25 print(line["time"], line["level"], line["msg"])
    26 for subtask in task["subtasks"]:
    27 print_logs(subtask)
    28
    29
    30print_logs(task)
  6. Run the script using Python 3:

    jovyan@mynotebook:~$ python3 tasklogs.py

Summary

You have successfully retrieved the logs of a Rok task.

What’s Next

Check out the rest of the maintenance operations that you can perform on your cluster.