Some hints on training with your own data

[kv2000]

Since our approach requires only the motion textures as input conditions, it is possible, and intuitive, to adapt it for different kinds of drivable human templates.

Assume that you have a skinned/drivable template mesh with a UV paradigm.
Since we have provided the tools in the training dataloader, that render the info attached on the vertexes to the textures, it would be intuitive to adapt it for training on other drivable human models with the following ingredients:

• The Canonical-pose vertex Position (referring to cached_ret_canonical_delta.pkl)
• The Posed vertex Position (referring to cached_ret_posed_delta.pkl)
• The Posed vertex Normal (referring to cached_temp_vert_normal.pkl)
• The Rotation and translation quaternions for each vertex (referring to cached_fin_rotation_quad.pkl, cached_fin_translation_quad.pkl)
• The Joint positions (referring to cached_joints.pkl)

[Yi-Cheng0101]

Hi, as you mentioned above, there are five files we need to prepare before training. Could you provide how you preprocess your original videos and use what kind of tools to get these five files. Thanks!

[hua-zi]

Maybe ASH_Train_Dataset.get_precomputed_human() in models/dataset_no_charactor_train.py can generate these files.

[hua-zi]

Hello, I'm interested in training ASH using my own multi-view video data. Could you please guide me on how to get these files similar to Subject0022_train.conf for my dataset? Thanks!

character{
    skeleton_dir =./meta_data/mydataset/ddc.skeleton                      # skeleton file
    skinning_dir =./meta_data/mydataset/ddc.skin                          # the skinning weights
    template_mesh_dir =./meta_data/mydataset/ddc.obj                      # the template meshes 
    graph_dir = ./meta_data/mydataset/ddc_graph.obj                       # the embedded graph, which is a lower res ones
    rest_pose_dir = ./meta_data/mydataset/ddc.motion                      # the rest pose 
    ...
    hand_mask_dir = ./meta_data/mydataset/hand_segmentation.txt           # for Dynacap Dataset characters we comment out this entry ;D 
}
dataset {
    ...
    # dofs
    skeleton_angles = ./meta_data/mydataset/skeletoolToGTPose/poseAngles.motion
    # rotation, and translation normalized dof; of (n, n-1, n-2 th frame) w.r.t the nth frame :D
    skeleton_angles_rotation_normalized = ./meta_data/mydataset/skeletoolToGTPose/poseAnglesRotationNormalized.motion
    ...
}

[Yi-Cheng0101]

Maybe ASH_Train_Dataset.get_precomputed_human() in models/dataset_no_charactor_train.py can generate these files.

Thanks! And do you know here I place my custom video and the command to run this?

[moqiyinlun]

But how can I get the "Subject0022/ddc.skin", "Subject0022/ddc.skeleton", "Subject0022/ddc.obj", "Subject0022/skeletoolToGTPose/poseAnglesRotationNormalized.motion", "Subject0022/skeletoolToGTPose/poseAngles.motion" for my own dataset, since I carefully reviewed the code and realized that generating those five cache files you mentioned requires the preparation of skeleton information in advance？