Skip to content

Commit

Permalink
Merge branch 'main' into resetoption
Browse files Browse the repository at this point in the history
  • Loading branch information
melindawangmsft authored Aug 8, 2023
2 parents e743e2c + 2f305e1 commit 8c91100
Show file tree
Hide file tree
Showing 39 changed files with 8,779 additions and 4 deletions.
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The content is converted to CMAF format with both a DASH and HLS manifest to sup
* Support for packaging VOD assets.
* Support for copying non-streamable assets.
* Marks migrated assets and provides HTML summary on analyze
* Support for statically encrypting the cotnent while packaging.
* Support for statically encrypting the content while packaging.

## Open Issues
* Live assets are not supported but will be in a future version of this tool.
Expand All @@ -66,6 +66,31 @@ You'll need to have the following permissions:
- The identity that runs this migration tool should be added to the ['Storage Blob Data Contributor'](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) role for the source and destination storage accounts.
- The identity used to migrate must have the role ['Key Vault Secrets Officer'](https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide?tabs=azure-cli#azure-built-in-roles-for-key-vault-data-plane-operations) on the key vault used to store the keys. If the key vault is not using Azure RBAC then you will have to create an Access policy giving secrets management permission to the identity used.

# Prerequisite dependencies

The following are the tools that we require the user to install on the machine that is running the migration tool

- [FFmpeg](https://ffmpeg.org/download.html)

Some types of AMS asset requires FFmpeg in order to complete (e.g. Smooth content).
To handle these type of contents, we expect that user install FFmpeg and add it to the path prior to running the migration tool so that it has access to FFmpeg.

Please follow info in this [link](https://ffmpeg.org/download.html) to install FFmpeg on yours system and add it to the path of your system.

Some installation examples:

- For Ubuntu, you can use the package manager 'apt' which will install and add ffmpeg to your path.

```
sudo apt install -y ffmpeg
```

- For Windows 11, you can use 'winget' to install from [gyan.dev](https://www.gyan.dev/ffmpeg/builds/) which will install and add ffmpeg to your path (Note that for windows, after you install, you need to recycle the command prompt windows to get the updated PATH.)

```
winget install ffmpeg
```

# Quick Start

## Build project
Expand All @@ -75,7 +100,7 @@ To build the project, you can run the following command
dotnet build
dotnet publish

the output binary will be in your publish folder (e.g. bin\Debug\net6.0\publish\AMSMigrate.exe for windows,
the output binary will be in your publish folder (e.g. bin\Debug\net6.0\publish\AMSMigrate.exe for windows,
bin\Debug\net6.0\publish\AMSMigrate for Linux)

There are two main commands: analyze and assets. You can view the help by running the following three commands
Expand Down Expand Up @@ -117,3 +142,7 @@ To analyze, you will use the analyze command, an example is
The html file and the log file location will be reported at the end of execution.

You can look at the help for analyze command to further customization.

# Deployment
## Running in Azure
The quickest way to run the tool in Azure is use the deployment scripts. See [deployment.md](deployment/deployment.md) for more instructions how to quickly run it.
2 changes: 1 addition & 1 deletion azure/AzureStorageUploader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ public async Task UploadAsync(
/// <param name="fileName">The output blob in the destination container.</param>
/// <param name="blob">The source blob.</param>
/// <param name="aesTransform">The optional AesTransform for source content decryption.</param>
/// <param name="cancellationToken">The cancellaton token for the async operation.</param>
/// <param name="cancellationToken">The cancellation token for the async operation.</param>
/// <returns></returns>
public async Task UploadBlobAsync(
string containerName,
Expand Down
2 changes: 1 addition & 1 deletion decryption/AssetDecryptor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
namespace AMSMigrate.Decryption
{
/// <summary>
/// Helper class to decrypt asset blobs when the asset was uploaded with storeage-encryption enabled.
/// Helper class to decrypt asset blobs when the asset was uploaded with storage-encryption enabled.
/// </summary>
public static class AssetDecryptor
{
Expand Down
158 changes: 158 additions & 0 deletions deployment/deployment.bicep
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
@description('The resource group of the media account')
param mediaAccountRG string

@description('Azure Media Services account name')
param mediaAccountName string

@description('The storage account where the migrated data is written')
param storageAccountName string

@description('The resource group of storage account where the migrated data is written')
param storageAccountRG string

@description('The region where the Azure media services account is present')
param location string = resourceGroup().location

@description('Set to true if you need to encrypt the content')
param encrypt bool = true

@description('The key vault to store the envcryption keys')
param keyvaultname string

@description('The resource group where key vault is present.')
param keyvaultRG string

@description('Additional command line arguments to pass')
param arguments array = []

var tags = {
name: 'azure-media-migration'
}

// The identity to create and the roles to assign.
var identifier = 'azure-media-migration'
var mediaRoleName = 'Media Services Media Operator'
var storageRoleName = 'Storage Blob Data Contributor'
var keyVaultRoleName = 'Key Vault Secrets Officer'

// Parameters for the container creation.
var cpus = 4
var memoryInGB = 16
var image = 'ghcr.io/azure/azure-media-migration:main'

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: 'azure-media-migration-identity'
location: location
tags: tags
}

resource mediaAccount 'Microsoft.Media/mediaservices@2023-01-01' existing = {
scope: resourceGroup(mediaAccountRG)
name: mediaAccountName
}

module mediaRoleAssignment 'roleassignment.bicep' = {
scope: resourceGroup(mediaAccountRG)
name: 'mediaRoleAssignement'
params: {
resourceName: mediaAccountName
principalId: managedIdentity.properties.principalId
roleName: mediaRoleName
storage: false
assignGroupRole: true
}
}

var storageAccountIds = map(mediaAccount.properties.storageAccounts, arg => arg.id)
module storageRoleAssignments 'storageaccounts.bicep' = {
name: 'storageRoleAssignments'
params: {
storageAccounts: storageAccountIds
principalId: managedIdentity.properties.principalId
storageRoleName: storageRoleName
}
}

resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' existing = {
name: storageAccountName
scope: resourceGroup(storageAccountRG)
}

module storageRoleAssignment 'roleassignment.bicep' = {
scope: resourceGroup(storageAccountRG)
name: 'storageRoleAssignment'
params: {
resourceName: storageAccountName
principalId: managedIdentity.properties.principalId
roleName: storageRoleName
storage: true
}
}

resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' existing = if (encrypt) {
name: keyvaultname
scope: resourceGroup(keyvaultRG)
}
module keyVaultRoleAssignment 'roleassignment.bicep' = if (encrypt) {
scope: resourceGroup(keyvaultRG)
name: 'keyVaultRoleAssignment'
params: {
resourceName: keyvaultname
principalId: managedIdentity.properties.principalId
roleName: keyVaultRoleName
storage: true
}
}

// Default arguments to the migration tool.
var defaultArguments = [
'dotnet'
'AMSMigrate.dll'
'assets'
'-s'
subscription().subscriptionId
'-g'
mediaAccountRG
'-n'
mediaAccountName
'-o'
storageAccount.properties.primaryEndpoints.blob
]

var encryptionArguments = [
'--encrypt-content'
'--key-vault-uri'
]

resource container 'Microsoft.ContainerInstance/containerGroups@2023-05-01' = {
name: identifier
tags: tags
location: location
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${managedIdentity.id}' : {}
}
}
properties: {
containers: [
{
name: 'amsmigrate'
properties: {
image: image
resources: {
requests: {
cpu: cpus
memoryInGB: memoryInGB
}
}
command: concat(defaultArguments, arguments, encrypt ? encryptionArguments : [], encrypt ? [ keyVault.properties.vaultUri ] : [])
}
}
]
osType: 'Linux'
restartPolicy: 'Never'
}
}

output follow string = 'az container logs -g ${resourceGroup().name} -n ${identifier} --follow'
31 changes: 31 additions & 0 deletions deployment/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Deploying the tool to the cloud.

## Create a Resource Group
Create it in the same region as the media services account being migrated.

```bash
az group create --location location --name migration
```

## Update the parameters.
The parameters for the deployment are in the file [parameters.bicepparam](parameters.bicepparam).
```bicep
// The media account being migrated.
param mediaAccountName = 'accountname'
param mediaAccountRG = 'resourcegroup'
// The storage account details where the migrated data is written.
param storageAccountName = 'storeagaccountname'
param storageAccountRG = 'storageresourcegroup'
```

## Deploy the resource.
```bash
az deployment group create --template-file deployment.bicep --resource-group migration --parameters parameters.bicepparam
```

## Monitor progress
While the ACI is running you can monitor the progress of the tool by running.
```bash
az container logs --follow -g migration -n azure-media-migration
```
22 changes: 22 additions & 0 deletions deployment/parameters.bicepparam
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
using './deployment.bicep'

// The media services account being migrated.
param mediaAccountName = 'provenanceuswc'
param mediaAccountRG = 'provenance'

// The storage account where migrated data is written.
param storageAccountName = 'amsencodermsitest'
param storageAccountRG = 'amsmediacore'

// setting to turn encryption on or off.

param encrypt = false
// The key vault to store encryption keys if encryption is turned on.
param keyvaultname = 'mpprovenance'
param keyvaultRG = 'provenance'

//additional arguments.
param arguments = [
'-t'
'$web/deployment/\${AssetName}'
]
64 changes: 64 additions & 0 deletions deployment/roleassignment.bicep
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
@description('The name of the resource to assign role')
param resourceName string

@description('The role to assign to the resource')
param roleName string

@description('The service principal to add the role assignment')
param principalId string

@description('true if the resource is storage else false')
param storage bool = false

@description('The role to assign to the resource group of the resource.')
param resourceGroupRoleName string = 'Reader'

@description('Assign a role to the resource group.')
param assignGroupRole bool = false

var roles = {
Contributor: '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c'
Reader: '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7'
'Storage Blob Data Contributor': '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe'
'Media Services Media Operator': '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/e4395492-1534-4db2-bedf-88c14621589c'
'Key Vault Secrets Officer':'/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/b86a8fe4-44ce-4948-aee5-eccb2c155cd7'
}

resource mediaAccount 'Microsoft.Media/mediaservices@2023-01-01' existing = if(!storage) {
name: resourceName
}

resource resourceGroupRoleNameAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (assignGroupRole) {
name: guid(resourceGroup().id, principalId)
scope: resourceGroup()
properties: {
roleDefinitionId: roles[resourceGroupRoleName]
principalId: principalId
principalType: 'ServicePrincipal'
}
}

resource mediaRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (!storage) {
name: guid(mediaAccount.id, principalId)
scope: mediaAccount
properties: {
roleDefinitionId: roles[roleName]
principalId: principalId
principalType: 'ServicePrincipal'
}
}

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' existing = if(storage) {
name: resourceName
}

resource storageRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = if(storage) {
name: guid(storageAccount.id, principalId)
scope: storageAccount
properties: {
roleDefinitionId: roles[roleName]
principalId: principalId
principalType: 'ServicePrincipal'
}
}

20 changes: 20 additions & 0 deletions deployment/storageaccounts.bicep
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
@description('The service principal to add the role assignment')
param principalId string

@description('Storage role name')
param storageRoleName string = 'Storage Blob Data Contributor'

@description('The storage accounts associated with the media account')
param storageAccounts array

module storageRoleAssignments './roleassignment.bicep' = [for storage in storageAccounts: {
name: 'storageRoleAssignment-${split(storage, '/')[8]}'
scope: resourceGroup(split(storage, '/')[4])
params: {
resourceName: split(storage, '/')[8]
roleName: storageRoleName
principalId: principalId
storage: true
}
}]

Loading

0 comments on commit 8c91100

Please sign in to comment.