美摄SDK For Android  3.14.0
Mask Generator 技术文档

Mask Generator文档

Mask Generator，中文意为蒙版。视频编辑技术的增强，需要精细化视频处理。视频技术增强的一个要求是加强界面部分区域的编辑与控制，强调细腻的视频处理，蒙版技术为局部精细化处理提供支持。

SDK提供了方便的编程接口，用户根据自己的需求，快捷实现Mask Generator。开发者可以根据客观需求，开发广阔应用。

蒙版包括字幕蒙版和形状蒙版，两者编程方式不同。形状蒙版需要用户定义区域，字幕蒙版不需要用户定义区域。后面主题会有介绍。下图是蒙版的两大组成部分。

形状蒙版包含：线性、镜面、圆形、矩形、心形，星形。蒙版整体构成如下：

1、编程原理

Mask Generator技术属于视频特效范围，通过视频特效进行控制。

首先开启Mask Generator特效；设定区域范围；应用蒙版特效。

Mask Generator属于对视频区域的精细化处理，必然对视频区域范围进行指定。对区域进行数学化的描述，是Mask Generator编程的难点与核心所在。

技术人员通过窗口的范围、蒙版层的范围、中心点、数学公式等因素，最终决定蒙版的区域。

2、SDK支持

下面接口参考api相关文档：Matrix、Path、PointF、Rect、RectF、NvsVideoFx、NvsMaskRegionInfo、NvsPosition2D、RegionInfo。

3、参数ID

通过参数ID，对蒙版属性进行控制。参数ID应用和AR Scene类似。

序号 参 数 类 型 最 大 值 最 小 值 默 认 值 说 明
001 Region Info ARBDATA 区域信息
002 Feather Width FLOAT 1000.0 0.0 0.0 羽化宽度
003 Keep RGB BOOL FALSE 保持RGB
004 Inverse Region BOOL FALSE 区域反转
005 Coordinate System STRING ndc 坐标类型
006 Feather Compensation BOOL True 羽化补偿
007 Text Mask Description String STRING 文本遮罩描述文件
008 Is Caption BOOL false 是否是字幕

4、NvsMaskReginInfo

应用Mask Generator的时间，核心是生成蒙版区域信息对象。也即NvsMaskRegionInfo对象。4.2主题中，会讲述典型区域对象的生成。

Mask Generator结构，参考下图。详细描述参考官网文档。

区域对象的生成，是坐标定位的结果。美摄SDK坐标系分为如下两种。

4.1、区域对象生成说明

第一步：生成NvsMaskReginInfo对象。

第二步：生成内部类ReginInfo对象。

第三步：给ReginInfo对象进行相关设置。

第四步：把ReginInfo对象注入到NvsMaskReginInfo对象中。

4.2、典型Mask区域NvsMaskRegionInfo对象生成（详细代码参考SdkDemo）

开发过程中，技术人员如果能确定坐标点精确位置，直接形成NvsMaskRegionInfo对象，来实现蒙版效果。

也可以通过导入的视频宽高，依据视频属性、中心点、蒙版层大小等因素，来计算精确位置。本文档案例就是根据相关属性进行定位。

本主题提供的技术思路，作为形状蒙版的技术参考。技术人员可以实现更加炫酷的蒙版效果。

4.2.1、线性蒙版

由于线性蒙版形状特点，相对简单。根据相关属性，得到点数组，根据点数组，形成蒙版对象。

下面函数通过视频中心点，蒙版区域宽高和角度形成蒙版边界点数组。

A、点数组形成
public static PointF[] buildLineMaskPoint(PointF center, int maskWidth, int maskHeight, float angle) {
PointF leftTopPoint = new PointF(center.x - maskWidth, center.y - maskHeight);
PointF rightTopPoint = new PointF(center.x + maskWidth, center.y - maskHeight);
PointF rightBottomPoint = new PointF(center.x + maskWidth, center.y);
PointF leftBottomPoint = new PointF(center.x - maskWidth, center.y);
return new PointF[]{getPointByAngle(leftTopPoint, center, angle), getPointByAngle(rightTopPoint, center, angle),
getPointByAngle(rightBottomPoint, center, angle), getPointByAngle(leftBottomPoint, center, angle)};
}
B、根据上面函数形成的点数组，及流媒体窗口相关信息与实际显示的宽高，生成线性蒙版区域信息对象。
public static NvsMaskRegionInfo buildPolygonMaskRegionInfo(PointF[] pointFList, NvsLiveWindow liveWindow, PointF size) {
List<NvsPosition2D> nvsPosition2DS = buildNvsPositionListFromPointFList(pointFList, liveWindow, size);
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_POLYGON);
//参数意义
regionInfo.setPoints(nvsPosition2DS);
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

4.2.2、镜面蒙版

根据相关属性，计算出点数组，根据点数组，生成蒙版区域信息对象

根据蒙版宽高，中心点，角度，生成区域点数组。

A、获取关键点数组
public static PointF[] buildMirrorMaskPoint(int maskWidth, PointF center, int maskHeight, float angle) {
PointF leftTopPoint = new PointF(center.x - maskWidth / 2, center.y - maskHeight / 2);
PointF rightTopPoint = new PointF(center.x + maskWidth / 2, center.y - maskHeight / 2);
PointF rightBottomPoint = new PointF(center.x + maskWidth / 2, center.y + maskHeight / 2);
PointF leftBottomPoint = new PointF(center.x - maskWidth / 2, center.y + maskHeight / 2);
return new PointF[]{getPointByAngle(leftTopPoint, center, angle), getPointByAngle(rightTopPoint, center, angle),
getPointByAngle(rightBottomPoint, center, angle), getPointByAngle(leftBottomPoint, center, angle)};
}
B、根据区域数组、流媒体窗口属性及蒙版实际宽高，生成蒙版区域信息对象。
public static NvsMaskRegionInfo buildPolygonMaskRegionInfo(PointF[] pointFList, NvsLiveWindow liveWindow, PointF size) {
List<NvsPosition2D> nvsPosition2DS = buildNvsPositionListFromPointFList(pointFList, liveWindow, size);
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_POLYGON);
//参数意义
regionInfo.setPoints(nvsPosition2DS);
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

4.2.3、圆形蒙版

根据相关属性，生成区域信息对象，生成蒙版区域信息对象，把区域信息对象注入到蒙版区域信息对象中。实现算法与流程参考下面代码。

public static NvsMaskRegionInfo buildCircleMaskRegionInfo(PointF center, float maskWidth, float maskHeight,
float angle, NvsLiveWindow liveWindow,
PointF size) {
float widthPercent = 0, heightPercent = 0;
widthPercent = maskWidth * 1.0f / size.x;
heightPercent = maskHeight * 1.0f / size.y;
center = mapViewToNormalized(center, liveWindow, size);
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_ELLIPSE2D);
//参数意义
regionInfo.setEllipse2D(new NvsMaskRegionInfo.Ellipse2D(new NvsPosition2D(center.x, center.y), widthPercent, heightPercent, 0));
NvsMaskRegionInfo.Transform2D transform2D = new NvsMaskRegionInfo.Transform2D();
transform2D.setRotation(-angle);
transform2D.setAnchor(new NvsPosition2D(center.x, center.y));
regionInfo.setTransform2D(transform2D);
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

4.2.4、矩形蒙版

根据相关属性，生成区域信息对象，生成蒙版区域信息对象，把区域信息对象注入到蒙版区域信息对象中。实现算法与流程参考如下代码。

public static NvsMaskRegionInfo buildRectMaskRegionInfo(PointF center, int width, int height,
float angle, NvsLiveWindow liveWindow,
float cornerRadiusRate, PointF size) {
// center = liveWindow.mapViewToNormalized(center);
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_CUBIC_CURVE);
//获取最小边 获取 圆角的半径
int minSize = width > height ? height : width;
int arcRadius = (int) (cornerRadiusRate * minSize * 0.5f);
int controlPointDis = (int) (arcRadius * 0.45);
//第一个点
PointF nextPoint1 = new PointF(center.x - width * 0.5f, center.y - height * 0.5f + controlPointDis);
PointF curPoint1 = new PointF(center.x - width * 0.5f, center.y - height * 0.5f + arcRadius);
PointF prePoint1 = new PointF(center.x - width * 0.5f, center.y + height * 0.5f - arcRadius);
Log.d(TAG, "point 1 c1.x:" + curPoint1.x + " y:" + curPoint1.y + " n1.x:" + nextPoint1.x
+ " y:" + nextPoint1.y + " p1.x:" + prePoint1.x + " y:" + prePoint1.y + " aD:" + arcRadius + "cD:" + controlPointDis);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint1, center, angle),
getPointByAngle(nextPoint1, center, angle), getPointByAngle(prePoint1, center, angle), liveWindow, size);
//第二点
PointF nextPoint2 = new PointF(center.x - width * 0.5f, center.y - height * 0.5f + arcRadius);
PointF curPoint2 = new PointF(center.x - width * 0.5f, center.y + height * 0.5f - arcRadius);
PointF prePoint2 = new PointF(center.x - width * 0.5f, center.y + height * 0.5f - controlPointDis);
Log.d(TAG, "point 2 c2.x:" + curPoint2.x + " y:" + curPoint2.y + " n2.x:" + nextPoint2.x
+ " y:" + nextPoint2.y + " p2.x:" + prePoint2.x + " y:" + prePoint2.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint2, center, angle),
getPointByAngle(nextPoint2, center, angle), getPointByAngle(prePoint2, center, angle), liveWindow, size);
//第三点
PointF nextPoint3 = new PointF(center.x - width * 0.5f + controlPointDis, center.y + height * 0.5f);
PointF curPoint3 = new PointF(center.x - width * 0.5f + arcRadius, center.y + height * 0.5f);
PointF prePoint3 = new PointF(center.x + width * 0.5f - arcRadius, center.y + height * 0.5f);
Log.d(TAG, "point 3 c3.x:" + curPoint3.x + " y:" + curPoint3.y + " n3.x:" + nextPoint3.x
+ " y:" + nextPoint3.y + " p1.x:" + prePoint3.x + " y:" + prePoint3.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint3, center, angle),
getPointByAngle(nextPoint3, center, angle), getPointByAngle(prePoint3, center, angle), liveWindow, size);
//第四点
PointF nextPoint4 = new PointF(center.x - width * 0.5f + arcRadius, center.y + height * 0.5f);
PointF curPoint4 = new PointF(center.x + width * 0.5f - arcRadius, center.y + height * 0.5f);
PointF prePoint4 = new PointF(center.x + width * 0.5f - controlPointDis, center.y + height * 0.5f);
Log.d(TAG, "point 4 c4.x:" + curPoint4.x + " y:" + curPoint4.y + " n4.x:" + nextPoint4.x
+ " y:" + nextPoint4.y + " p4.x:" + prePoint4.x + " y:" + prePoint4.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint4, center, angle),
getPointByAngle(nextPoint4, center, angle), getPointByAngle(prePoint4, center, angle), liveWindow, size);
//第五点
PointF nextPoint5 = new PointF(center.x + width * 0.5f, center.y + height * 0.5f - controlPointDis);
PointF curPoint5 = new PointF(center.x + width * 0.5f, center.y + height * 0.5f - arcRadius);
PointF prePoint5 = new PointF(center.x + width * 0.5f, center.y - height * 0.5f + arcRadius);
Log.d(TAG, "point 5 c5.x:" + curPoint5.x + " y:" + curPoint5.y + " n5.x:" + nextPoint5.x
+ " y:" + nextPoint5.y + " p5.x:" + prePoint5.x + " y:" + prePoint5.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint5, center, angle),
getPointByAngle(nextPoint5, center, angle), getPointByAngle(prePoint5, center, angle), liveWindow, size);
//第六点
PointF nextPoint6 = new PointF(center.x + width * 0.5f, center.y + height * 0.5f - arcRadius);
PointF curPoint6 = new PointF(center.x + width * 0.5f, center.y - height * 0.5f + arcRadius);
PointF prePoint6 = new PointF(center.x + width * 0.5f, center.y - height * 0.5f + controlPointDis);
Log.d(TAG, "point 6 c6.x:" + curPoint6.x + " y:" + curPoint6.y + " n6.x:" + nextPoint6.x
+ " y:" + nextPoint6.y + " p6.x:" + prePoint6.x + " y:" + prePoint6.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint6, center, angle),
getPointByAngle(nextPoint6, center, angle), getPointByAngle(prePoint6, center, angle), liveWindow, size);
//第七点
PointF nextPoint7 = new PointF(center.x + width * 0.5f - controlPointDis, center.y - height * 0.5f);
PointF curPoint7 = new PointF(center.x + width * 0.5f - arcRadius, center.y - height * 0.5f);
PointF prePoint7 = new PointF(center.x - width * 0.5f + arcRadius, center.y - height * 0.5f);
Log.d(TAG, "point 7 c7.x:" + curPoint7.x + " y:" + curPoint7.y + " n7.x:" + nextPoint7.x
+ " y:" + nextPoint7.y + " p1.x:" + prePoint7.x + " y:" + prePoint7.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint7, center, angle),
getPointByAngle(nextPoint7, center, angle), getPointByAngle(prePoint7, center, angle), liveWindow, size);
//第八点
PointF nextPoint8 = new PointF(center.x + width * 0.5f - arcRadius, center.y - height * 0.5f);
PointF curPoint8 = new PointF(center.x - width * 0.5f + arcRadius, center.y - height * 0.5f);
PointF prePoint8 = new PointF(center.x - width * 0.5f + controlPointDis, center.y - height * 0.5f);
Log.d(TAG, "point 8 c8.x:" + curPoint8.x + " y:" + curPoint8.y + " n8.x:" + nextPoint8.x
+ " y:" + nextPoint8.y + " p8.x:" + prePoint8.x + " y:" + prePoint8.y);
maskRegionInfoAddPoints(regionInfo, getPointByAngle(curPoint8, center, angle),
getPointByAngle(nextPoint8, center, angle), getPointByAngle(prePoint8, center, angle), liveWindow, size);
Log.d(TAG, "point ====================================================================================");
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

4.2.5、爱心蒙版

依据相关属性，计算点数组，根据点数组，生成区域信息对象，生成蒙版信息对象，把区域对象注入到蒙版信息对象中。具体实现算法与流程参考下面代码。

public static NvsMaskRegionInfo buildHeartMaskRegionInfo(PointF center, int radius, float angle,
NvsLiveWindow liveWindow, PointF size) {
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
// 贝塞尔曲线MASK_REGION_TYPE_CUBIC_CURVE+
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_CUBIC_CURVE);
PointF topIntersectionPoint = new PointF(center.x, center.y - radius * (2 * 1.0f / 6));
PointF bottomIntersectionPoint = new PointF(center.x, center.y + radius);
PointF prePoint = getPointByAngle(new PointF(center.x + 5 * 1.0f / 7 * radius, center.y - 0.8f * radius), center, angle);
PointF curPoint = getPointByAngle(topIntersectionPoint, center, angle);
PointF nextPoint = getPointByAngle(new PointF(center.x - 5 * 1.0f / 7 * radius, center.y - 0.8f * radius), center, angle);
PointF prePoint1 = getPointByAngle(new PointF(center.x - 16 * 1.0f / 13 * radius, center.y + 0.1f * radius), center, angle);
PointF curPoint1 = getPointByAngle(bottomIntersectionPoint, center, angle);
PointF nextPoint1 = getPointByAngle(new PointF(center.x + 16 * 1.0f / 13 * radius, center.y + 0.1f * radius), center, angle);
PointF[] pointFS = new PointF[]{curPoint, nextPoint, prePoint, curPoint1, nextPoint1, prePoint1};
List<NvsPosition2D> nvsPosition2DS = buildNvsPositionListFromPointFList(pointFS, liveWindow, size);
regionInfo.setPoints(nvsPosition2DS);
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

4.2.6、星形蒙版

依据相关属性，计算出点数组，根据点数组，生成区域信息对象，生成蒙版信息对象，把区域对象注入到蒙版信息对象中。具体实现算法与流程参考如下代码。

public static NvsMaskRegionInfo buildStarMaskRegionInfo(PointF center, int width, float rotation, NvsLiveWindow liveWindow, PointF size) {
//蒙版特效区域信息
NvsMaskRegionInfo nvsMaskRegionInfo = new NvsMaskRegionInfo();
//设置类型
// 椭圆 MASK_REGION_TYPE_ELLIPSE2D
// 多边形 MASK_REGION_TYPE_POLYGON
// 贝塞尔曲线MASK_REGION_TYPE_CUBIC_CURVE+
NvsMaskRegionInfo.RegionInfo regionInfo = new NvsMaskRegionInfo.RegionInfo(NvsMaskRegionInfo.MASK_REGION_TYPE_POLYGON);
//外圆
float radius = width / 2.0f;
float angel = (float) (Math.PI * 2 / 5);
PointF[] outPoints = new PointF[5];
//这里是获取五角星的五个定点的坐标点位置
for (int i = 1; i < 6; i++) {
float x = (float) (center.x - Math.sin(i * angel) * radius);
float y = (float) (center.y - Math.cos(i * angel) * radius);
outPoints[i - 1] = new PointF(x, y);
}
float radiusRate = 0.5f; //2/5
//内圆
float internalRadius = radius * radiusRate;
float internalAngel = (float) (Math.PI * 2 / 5);
PointF[] inPoints = new PointF[5];
//这里是获取五角星的五个定点的坐标点位置
for (int i = 1; i < 6; i++) {
float x = (float) (center.x - Math.sin(i * internalAngel + Math.PI / 2 - Math.PI * 3 / 10) * internalRadius);
float y = (float) (center.y - Math.cos(i * internalAngel + Math.PI / 2 - Math.PI * 3 / 10) * internalRadius);
inPoints[i - 1] = new PointF(x, y);
}
//加入到一个集合中 一外一内的顺序
PointF[] allPoints = new PointF[10];
for (int i = 0; i < 5; i++) {
PointF out = getPointByAngle(outPoints[i], center, rotation);
PointF in = getPointByAngle(inPoints[i], center, rotation);
allPoints[i * 2] = out;
allPoints[i * 2 + 1] = in;
}
List<NvsPosition2D> position2DList = buildNvsPositionListFromPointFList(allPoints, liveWindow, size);
regionInfo.setPoints(position2DList);
nvsMaskRegionInfo.addRegionInfo(regionInfo);
return nvsMaskRegionInfo;
}

5、根据区域信息，进行设置

环境搭建、对象生成、视频特效对象生成、区域设置、特效添加。

5.1、蒙版设置

生成特效对象 

maskFx = videoClip.appendRawBuiltinFx(NvsConstants.KEY_MASK_GENERATOR);

5.1.1、设置方式

SDK对蒙版的设置有两种方式，本质一样。实际应用中，两种方式可以混用。

5.1.1.1、参数调整

常规方式是通过参数调整。示例代码如下：

maskFx.setIntVal(“参数ID”,相应值);
maskFx.setFloatVal(“Feather Width”, 相应值);
maskFx.setStringVal(“Coordinate System”, 相应值);
maskFx.setBooleanVal(“Keep RGB”,相应值);
maskFx.setArbDataVal(“Region Info”,相应值);

5.1.1.2、函数方式

也可以通过相关函数进行硬编码。示例代码如下：

//设置区域羽化宽度，参数自己调节
maskFx.setInverseRegion(false);
//是否忽略背景
maskFx.setIgnoreBackground(true);
//设置是否为局部
maskFx.setRegional(true);
maskFx.setRegionInfo(maskRegionInfo);

5.1.2、参考代码

if (maskFx != null) {//如果找到蒙版特效
//设置区域羽化宽度，参数自己调节
maskFx.setRegionalFeatherWidth(10);
//设置区域反转，根据需要进行设置
maskFx.setInverseRegion(false);
//是否忽略背景
maskFx.setIgnoreBackground(true);
//设置是否为局部
maskFx.setRegional(true);
//如果是字幕蒙版
if (infoData.getMaskType() == MaskInfoData.MaskType.TEXT) {
maskFx.setStringVal(NvsConstants.KEY_MASK_STORYBOARD_DESC, infoData.getTextStoryboard());
//区域信息数据设置为空
maskFx.setRegionInfo(null);
}
//如果不是字幕蒙版
else {
//故事板描述设置为空
maskFx.setStringVal(NvsConstants.KEY_MASK_STORYBOARD_DESC, "");
//设置区域信息数据
maskFx.setRegionInfo(infoData.getMaskRegionInfo());
}
}

5.2、特别说明

5.2.1、非字幕蒙版

故事板描述设置为空字符串，区域信息设置具体的值。

代码示例：

maskFx.setStringVal(NvsConstants.KEY_MASK_STORYBOARD_DESC, "");
//蒙版区域信息对象，4.2主题讲述过该对象的生成
NvsMaskRegionInfo nvsMaskRegionInfo;
//根据蒙版区域信息对象，应用蒙版
maskFx.setRegionInfo(nvsMaskRegionInfo);

5.2.2、字幕蒙版

字幕蒙版相对简单，不需要生成蒙版区域对象和区域对象。只需要进行NvsConstants.*KEY_MASK_STORYBOARD_DESC参数ID设置就可以。*

给故事板描述设置具体字符串，区域信息设置为空。

代码示例：

String captionInfo = “美摄字幕蒙版测试字符串”;
maskFx.setStringVal(NvsConstants.KEY_MASK_STORYBOARD_DESC, captionInfo);
maskFx.setRegionInfo(null);
制作者   doxygen 1.8.17