MeiCam SDK For Android  3.14.2
Quick Access Guide

NveEffectKit User Guide


NveEffectKit integrates the functions of beauty, beauty shape, makeup, filters, and face props. The dependent library is the Meishe Effect library 3.8.0 or above.

Quick Start

  • Meishe Download the Face Version of the SDK from the Official Website.

    Download the SDK package: NvStreamingSdk_Android_..._ARSCENE_MS.zip

  • Open ./samples/android/NvEffectKitDemo with Android Studio to quickly try out the related features.
  • You can refer to the FAQ while running the demo.

Importing the Project?

Android is packaged as an AAR library(NveEffectKit.aar), copied to the libs directory(./app/libs), and the compilation and import of aar is set in build.gradle >NveEffectKit.aar path: ./samples/android/NvEffectKitDemo/libBase/libs/NveEffectKit.aar

api fileTree(dir: 'libs', include: ['*.jar', '*.aar'])

Initialize the SDK

You need to pass in the authorization file path, which can be the path under assets or the local absolute path. effectsdkdemo.licPlease contact Meishe Sales to obtain.

String licPath = "assets:/effectsdkdemo.lic";
NveEffectKit.getInstance().init(this, licPath);

Initialize models such as face detection

To initialize the model, you need to pass in the model type and path (the path can be the assets path or the local absolute path)

NveEffectKit.getInstance().initModel(this, NveDetectionModelType.FACE, faceModelPath);
Type Name Use
NveDetectionModelType.FACE Face Use of face props
NveDetectionModelType.FAKE_FACE FakeFace Use of face props
NveDetectionModelType.MAKEUP2 Makeup Makeup use
NveDetectionModelType.FACE_COMMON FaceCommon Beauty and Makeup use
NveDetectionModelType.ADVANCED_BEAUTY Beauty Beauty and Makeup use
NveDetectionModelType.EYEBALL Eyeball Some face props will use eyeball effects
NveDetectionModelType.SEGMENTATION Background segmentation Some face props will use segmentation effects
NveDetectionModelType.AVATAR Avatar The special effect category that moves according to the expression will use this model
  • NveDetectionModelType.MAKEUP2 has been deprecated since SDK version 3.13.x and above.
  • NveDetectionModelType.FACE_COMMON and NveDetectionModelType.ADVANCED_BEAUTY require a minimum SDK version of 3.13.x.
  • For details on the face model, refer to the demo code.

    samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/Constants.java

    samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/MainActivity.java
    private void initEffectModel() {}
  • For more information FAQ.

For camera preview and recording, refer to the demo code.

samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/camera/...

samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/MainActivity.java
private GLSurfaceView mGlView;
private RenderCameraUtil mRenderCameraUtil;

Beauty

//Use of beauty
NveBeauty beauty= new NveBeauty();
//The switch can be controlled by the enable parameter
beauty.setEnable(true);
//It only needs to be set once, and subsequent modification parameters can directly call the parameter set method of beauty
NveEffectKit.getInstance().setBeauty(beauty);
//There are many types of dermabrasion and whitening, the default types are as follows
beauty.setBlurType(NveBeautyBlurTypeEnum.AdvancedBuffingSkin1);
beauty.setWhiteningType(NveBeautyWhiteningTypeEnum.WhiteningA);
Parameter Type Use
enable boolean Beauty function switch
blurType NveBeautyBlurTypeEnum Microdermabrasion type
blurIntensity float Microdermabrasion intensity,[0-1]
whiteningType NvBeautyWhiteningTypeEnum Whitening type
whiteningIntensity float Whitening strength,[0-1]
matteIntensity float Degreasing intensity,[0-1]
matteFillRadius float Degreasing Radius,[0-1]
reddeningIntensity float Ruddy intensity,[0-1]
definitionIntensity float Definition intensity,[0-1]
sharpnessIntensity float Sharpness intensity,[0-10]

Microdermabrasion type

Name Use
NveBeautyBlurTypeEnum.BuffingSkin Ordinary microdermabrasion
NveBeautyBlurTypeEnum.AdvancedBuffingSkin1 Advanced microdermabrasion 1
NveBeautyBlurTypeEnum.AdvancedBuffingSkin2 Advanced microdermabrasion 2
NveBeautyBlurTypeEnum.AdvancedBuffingSkin3 Advanced microdermabrasion 3
  • For more information FAQ.

Whitening type

Name Use
NvBeautyWhiteningTypeEnum.WhiteningA WhiteningA
NvBeautyWhiteningTypeEnum.WhiteningB WhiteningB
  • For more information FAQ.

Beauty Shape

//Use Beauty Shape
NveShape shape= new NveShape();
shape.setEnable(true);
NveEffectKit.getInstance().setShape(shape);
//How to Use the Beauty Package?
//Taking face slimming as an example
//Add a new beauty model and pass in the type separately.
float shapeDegree = 1f;
//packagePath This is a placeholder path; replace it with the actual resource package path.
String packagePath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh";
// Resource package license file path.
String packageLicPath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.lic";
// If the resource package file ends with .facemesh, the type is ASSET_PACKAGE_TYPE_FACE_MESH; if it is a warp package, the type is ASSET_PACKAGE_TYPE_WARP.
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh
int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_FACE_MESH;
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.warp
// int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_WARP;
//shapePackageID:Beauty Package ID (The package needs to be installed; see the installation instructions below)
String shapePackageID = NveAssetPackageManagerUtil.installFxPackage(packagePath, packageLicPath, packageType);
shape.setShapeData(NveShapeTypeEnum.FACE_SIZE, new NveShapeData(shapeDegree, shapePackageID));
//Modify beauty intensity
shape.setDegreeByType(NveShapeTypeEnum.FACE_SIZE, shapeDegree);
NveEffectKit.getInstance().setShape(shape);
Name Use
NveShapeTypeEnum.FACE_WIDTH Narrow face,[-1,1]
NveShapeTypeEnum.FACE_LENGTH Small face,[-1,1]
NveShapeTypeEnum.FACE_SIZE Face slimming,[-1,1]
NveShapeTypeEnum.FORE_HEAD Forehead,[-1,1]
NveShapeTypeEnum.CHIN Jaw,[-1,1]
NveShapeTypeEnum.EYE_SIZE Big eyes,[-1,1]
NveShapeTypeEnum.EYE_ANGLE Corner of eye,[-1,1]
NveShapeTypeEnum.NOSE_WIDTH Thin nose,[-1,1]
NveShapeTypeEnum.NOSE_LENGTH Proboscis,[-1,1]
NveShapeTypeEnum.MOUTH_SIZE Mouth type,[-1,1]
NveShapeTypeEnum.MOUTH_CORNER Corner of mouth,[-1,1]

Micro plastic surgery

//Use Micro plastic surgery
NveMicroShape microShape = new NveMicroShape();
microShape.setEnable(true);
NveEffectKit.getInstance().setMicroShape(microShape);
//Add a new beauty model and pass in the type separately.
float shapeDegree = 0.5f;
//packagePath This is a placeholder path; replace it with the actual resource package path.
String packagePath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh";
// Resource package license file path.
String packageLicPath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.lic";
// If the resource package file ends with .facemesh, the type is ASSET_PACKAGE_TYPE_FACE_MESH; if it is a warp package, the type is ASSET_PACKAGE_TYPE_WARP.
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh
int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_FACE_MESH;
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.warp
// int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_WARP;
//shapePackageID:Beauty Package ID (The package needs to be installed; see the installation instructions below)
String shapePackageID = NveAssetPackageManagerUtil.installFxPackage(packagePath, packageLicPath, packageType);
//Taking face slimming as an example
shape.setMicroShapeData(NveMicroShapeTypeEnum.HEAD_SIZE, new NveMicroShapeData(shapeDegree,shapePackageID));
//Modify micro-shaping intensity
shape.setDegreeByType(NveMicroShapeTypeEnum.HEAD_SIZE, shapeDegree);
Name Use
NveMicroShapeTypeEnum.HEAD_SIZE Shrink head,[-1,1]
NveMicroShapeTypeEnum.CHEEKBONE_WIDTH Cheekbones wide,[-1,1]
NveMicroShapeTypeEnum.JAW_WIDTH Jaw width,[-1,1]
NveMicroShapeTypeEnum.TEMPLE_WIDTH Temple width,[-1,1]
NveMicroShapeTypeEnum.NASOLABIAL_FOLDS Nasal pattern,[0,1]
NveMicroShapeTypeEnum.DARK_CIRCLES Dark circles,[0,1]
NveMicroShapeTypeEnum.BRIGHTEN_EYES Eye-catching,[0,1]
NveMicroShapeTypeEnum.WHITEN_TEETH Beautiful teeth,[0,1]
NveMicroShapeTypeEnum.EYE_ANGLE_DEGREE Eye angle,[-1,1]
NveMicroShapeTypeEnum.EYE_DISTANCE Eyes apart,[-1,1]
NveMicroShapeTypeEnum.EYE_HEIGHT Eye height,[-1,1]
NveMicroShapeTypeEnum.EYE_WIDTH Eye width,[-1,1]
NveMicroShapeTypeEnum.EYE_ARC Eye curvature,[-1,1]
NveMicroShapeTypeEnum.EYE_Y_OFFSET Eyes up and down,[-1,1]
NveMicroShapeTypeEnum.PHILTRUM Among people,[-1,1]
NveMicroShapeTypeEnum.NOSE_BRIDGE_WIDTH Nose bridge width,[-1,1]
NveMicroShapeTypeEnum.NOSE_HEAD Nose,[-1,1]
NveMicroShapeTypeEnum.EYEBROW_THICKNESS Eyebrow thickness,[-1,1]
NveMicroShapeTypeEnum.EYEBROW_ANGLE Eyebrow angle,[-1,1]
NveMicroShapeTypeEnum.EYEBROW_Y_OFFSET Eyebrow up and down,[-1,1]
NvMicroShapeTypeEnum.EYEBROW_X_OFFSET Brow distance,[-1,1]

Meishe SDK For Android: List of effects names (meishesdk.com)

The various parameters of beauty, beauty and micro-shaping are from the descriptions of AR Scene in the beauty document warehouse. For more advanced usage, see the link above.


Makeup

//Use Makeup
NveMakeup makeUp= new NveMakeup();
makeUp.setEnable(true);
NveEffectKit.getInstance().setMakeup(makeUp);
//NvSingleMakeUp comes from the makeup.json parsed in the makeup bag
//(The package needs to be installed, installation see the installation below)
makeUp.addSingleMakeUp(NveMakeupTypeEnum, NveSingleMakeUp);
//Remove the makeup used according to the type of makeup, and only one of each type of makeup can be added, such as only one type of lipstick.
makeUp.removeSingleMakeUp(NveMakeupTypeEnum);
//For the use of makeup, NvComposeMakeup comes from the info.json parsed in the makeup pack
//(The package needs to be installed, installation see the installation below)
makeUp.setComposeMakeup(NveComposeMakeup);
//The removal of makeup. Makeup is composed of multiple single makeup, beauty, beauty, micro-plastic, filter, etc., and can only apply one makeup at a time
makeUp.removeComposeMakeUp();
//Adjust the intensity of single makeup
makeUp.setIntensityByType(NveMakeupTypeEnum.Brighten,1);
//Adjust the color of single makeup
makeUp.setColorByType(NveMakeupTypeEnum.Brighten,new NvsColor(0,0,0,0));

For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData

Name Use
NveMakeupTypeEnum.Lip Lipstick
NveMakeupTypeEnum.Eyebrow Eyebrow
NveMakeupTypeEnum.Eyeshadow Eye shadow
NveMakeupTypeEnum.Eyelash Eyelash
NveMakeupTypeEnum.Eyeliner Informer
NveMakeupTypeEnum.Blusher Blush
NveMakeupTypeEnum.Shadow Contour
NveMakeupTypeEnum.Brighten Highlights
NveMakeupTypeEnum.ComposeMakeup Makeup

Filter

//Must pass ID, package filter needs to be installed, pass in the ID after installation (installation see installation below)
NveFilter filter = new NveFilter(effectId);
NveEffectKit.getInstance().addFilter(filter);
//setFilter means that with the filter, other parameter modification operations can be performed later, such as modifying the intensity:
filter.setFilterIntensity(0.5f);
//Remove filter
NveEffectKit.getInstance().removeFilter(filter);

For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/FilterJsonParseData

Prop

NveFaceProp faceProp = new NveFaceProp();
faceProp.setEnable(true);
NveEffectKit.getInstance().setFaceProp(faceProp);
//Set the prop Id(the prop pack needs to be installed, see installation below)
String propId = "xxx"
faceProp.setPropId(propId);

For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/StickerJsonParseData

Custom effects

NveEffectKit In addition to the above existing features, users can create their own Meisak-supported effects, using the middle layer to render. For example, the background segmentation function (need to initialize the face and segmentation model in advance) is implemented as follows

//Create background segmentation
NvsRational aspectRatio = new NvsRational(9, 16);
NvsVideoEffect mSegmentationEffect = NvsEffectSdkContext.getInstance().createVideoEffect("Segmentation Background Fill", aspectRatio);
mSegmentationEffect.setIntVal("Stretch Mode", 2);
mSegmentationEffect.setMenuVal("Segment Type", "Half Body");
mSegmentationEffect.setColorVal("Background Color", new NvsColor(0, 1, 0, 1));
//Add render
NveEffectKit.getInstance().addEffect(mSegmentationEffect);
//Add render according to index
NveEffectKit.getInstance().insertEffect(mSegmentationEffect,0);
//Remove created effects
NveEffectKit.getInstance().removeEffect(mSegmentationEffect);

Package installation

The above contents basically involve package installation, and installation instructions are available on the official website

Meishe SDK For Android: Material pack installation (meishesdk.com)

Can use has good tools (NvAssetPackageManagerUtil) sample code:

//packagePath This is a placeholder path; replace it with the actual resource package path.
String packagePath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh";
// Resource package license file path.
String packageLicPath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.lic";
// If the resource package file ends with .facemesh, the type is ASSET_PACKAGE_TYPE_FACE_MESH; if it is a warp package, the type is ASSET_PACKAGE_TYPE_WARP.
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh
int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_FACE_MESH;
//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.warp
// int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_WARP;
//shapePackageID
String shapePackageID = NveAssetPackageManagerUtil.installFxPackage(packagePath, packageLicPath, packageType);

The parameters that need to be paid attention to are:

1、packagePath– packagePath– The path to the installation package, which can be the assets path or the local absolute path

2、packageLicPath– Specifies the license file path of the installation package

3、packageType– Installation package type. For details, see:Meishe SDK For Android: com.meicam.sdk.NvsAssetPackageManager Class reference (meishesdk.com)


Render

1.RenderCore Correlation initialization

//Note: The SDK must be initialized in advance
NveEffectKit.getInstance().initialize();
2.NveRenderInput Instructions

NveRenderInput mainly includes rendering configuration (NveRenderConfig), texture (NveTexture), Buffer (NveImageBuffer), image format (pixelFormat). The specific parameters are as follows:

(1)NveRenderConfig parameters, including camera orientation(cameraOrientation) and render mode (NveRenderMode)

NveRenderMode Instructions
NveRenderConfig.NveRenderMode.TEXTURE_TEXTURE Texture in, texture out
NveRenderConfig.NveRenderMode.BUFFER_TEXTURE Buffer in, texture out
NveRenderConfig.NveRenderMode.BUFFER_BUFFER Buffer in, Buffer out

(2)NveTexture Correlation parameter

Argument Instructions
textureId Texture ID
mirror Whether to flip horizontally, OES Texture to Texture 2D texture required
NveSize Texture width and height
NveTextureType Texture type
NveTextureLayout Texture direction
NveTextureType Instructions
NveTextureType.NV_COMMON_TEXTURE Ordinary texture
NveTextureType.NV_OES_TEXTURE OpenGL OES texture
NveTextureLayout Instructions
NveTextureLayout.UP_SIDE_DOWN Upside down
NveTextureLayout.DOWN_SIDE_UP Normal

(3)NveImageBuffer Correlation parameter

Name Instructions
data Buffer data
NveSize Buffer width and height
mirror Whether horizontal flip
displayRotation The Angle that the current image needs to be rotated
rowPitchY The number of bytes per line of the image in the Frame, if the image format is NV21,YUV420, refers to the Y plane
rowPitchUV The number of bytes per line of the image in the Frame is generally referred to as the UV plane, which is meaningless if the format is RGBA

(4)pixelFormat Image format, which is encapsulated in NvFormatType. For details, see the official websiteVideo frame pixel format

3.NveRenderOutput Instructions

NveRenderOutput mainly includes the render result (NveRenderError), the output texture (NveTexture, parameters are the same as above), and the output Buffer (NveImageBuffer, parameters are the same as above). The render result parameters are as follows

Name Instructions
NveRenderError.NO_ERROR Successful
NveRenderError.ERROR_UNKNOWN Unknown error
NveRenderError.INVALID_TEXTURE Invalid texture
NveRenderError.INVALID_INPUT_PARAM Invalid input parameter for NvRenderInput

4.render

//Input configuration
NveRenderInput mRenderInput = new NveRenderInput();
mRenderInput.setPixelFormat(NveFormatType.VIDEO_FRAME_PIXEL_FROMAT_NV21.getType());
NveRenderConfig config = new NveRenderConfig()
.setRenderMode(NveRenderConfig.NveRenderMode.TEXTURE_TEXTURE)
.setCameraOrientation(mCameraProxy.getOrientation()).build();
mRenderInput.setConfig(config);
//Input texture
NveTexture nvTexture = new NveTexture();
nvTexture.setTextureId(displayTex);
Texture.setTextureType(NveTextureType.NV_OES_TEXTURE);
nvTexture.setSize(new NveSize(texWidth, texHeight));
nvTexture.setMirror(mCameraProxy.isFlipHorizontal());
mRenderInput.setTexture(nvTexture);
//Input Buffer
NveImageBufferbuffer = new NveImageBuffer();
buffer.setData(data);
buffer.setSize(new NveSize(bufferWidth, bufferHeight));
buffer.setDisplayRotation(mOrientation);
buffer.setMirror(mCameraProxy.isFlipHorizontal());
mRenderInput.setImageBuffer(buffer);
//render
NveRenderOutput renderOutput = mNvEffectKit.renderEffect(mRenderInput);
......
if (null != renderOutput) {
//recycle render out
NveEffectKitt.recycleOutput(renderOutput);
}

5.Resource release

When exiting or no longer need to render, you need to release the relevant resources, you can call the following method

NveEffectKit.getInstance().cleanUp();

Note: This method needs to be called on the thread where GLContext is located

Destroy

To destroy the SDK in the case of page exit or app kill, call the following method

NveEffectKit.getInstance().destroy();

Note: If you destroy a page when it exits, then you need to use the SDK elsewhere, so you need to re-initialize the SDK before using it, that is, call the init method

FAQ

1.When using a newer version of Android Studio, running the APK may fail. In this case, you need to change the Gradle JDK version to JDK 11.

>Settings->Build,Execution,Deployment->Build Tools->Gradle->Gradle JDK (java 11)

Example of a similar error:

FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':app:processDebugMainManifest'.
> Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @16a1c82d
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @16a1c82d

2.What is the difference between face240 and face106 in the model?

240 and 106 refer to the number of facial landmark points; 240 provides better results.Makeup is not supported with 160-point landmarks.

3.What is the difference between Standard Skin Smoothing, Advanced Skin Smoothing 1, 2, and 3?

Advanced Skin Smoothing 1 is the standard effect for Android, Advanced Skin Smoothing 2 is the standard effect for iOS, and Advanced Skin Smoothing 3 is optimized for male subjects.

See documentation:AR Scene Advanced Beauty Type.

4.What is the difference between Beauty A and Beauty B?

They use different implementation methods; Beauty B uses a LUT file. See documentation:AR Scene Whitening Lut File.

5. Is custom GLSurfaceView/camera preview supported?

Sample code is provided for reference. If you already have your own implementation, you can provide the texture or buffer data to be rendered. For more details, please contact Meishe Technical Support.

6.No rendering effect or black screen after importing the project?

Check whether the lic file in your project is correctly bound to your applicationId. For more details, please contact Meishe Sales.

7.How to resolve conflicts after integrating NvAndroidStreamingSdk.jar?

Remove the existing NvAndroidStreamingSdk.jar from your project and use NveEffectKit.aar instead.

com