MeiCam SDK For Android
3.14.2
|
NveEffectKit integrates the functions of beauty, beauty shape, makeup, filters, and face props. The dependent library is the Meishe Effect library 3.8.0 or above.
Download the SDK package:
NvStreamingSdk_Android_..._ARSCENE_MS.zip
./samples/android/NvEffectKitDemo
with Android Studio
to quickly try out the related features.Android is packaged as an AAR library(NveEffectKit.aar
), copied to the libs directory(./app/libs
), and the compilation and import of aar is set in build.gradle >NveEffectKit.aar
path: ./samples/android/NvEffectKitDemo/libBase/libs/NveEffectKit.aar
You need to pass in the authorization file path, which can be the path under assets or the local absolute path. effectsdkdemo.lic
Please contact Meishe Sales to obtain.
To initialize the model, you need to pass in the model type and path (the path can be the assets path or the local absolute path)
Type | Name | Use |
---|---|---|
NveDetectionModelType.FACE | Face | Use of face props |
NveDetectionModelType.FAKE_FACE | FakeFace | Use of face props |
NveDetectionModelType.MAKEUP2 | Makeup | Makeup use |
NveDetectionModelType.FACE_COMMON | FaceCommon | Beauty and Makeup use |
NveDetectionModelType.ADVANCED_BEAUTY | Beauty | Beauty and Makeup use |
NveDetectionModelType.EYEBALL | Eyeball | Some face props will use eyeball effects |
NveDetectionModelType.SEGMENTATION | Background segmentation | Some face props will use segmentation effects |
NveDetectionModelType.AVATAR | Avatar | The special effect category that moves according to the expression will use this model |
samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/Constants.java
samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/camera/...
Parameter | Type | Use |
---|---|---|
enable | boolean | Beauty function switch |
blurType | NveBeautyBlurTypeEnum | Microdermabrasion type |
blurIntensity | float | Microdermabrasion intensity,[0-1] |
whiteningType | NvBeautyWhiteningTypeEnum | Whitening type |
whiteningIntensity | float | Whitening strength,[0-1] |
matteIntensity | float | Degreasing intensity,[0-1] |
matteFillRadius | float | Degreasing Radius,[0-1] |
reddeningIntensity | float | Ruddy intensity,[0-1] |
definitionIntensity | float | Definition intensity,[0-1] |
sharpnessIntensity | float | Sharpness intensity,[0-10] |
Microdermabrasion type
Name | Use |
---|---|
NveBeautyBlurTypeEnum.BuffingSkin | Ordinary microdermabrasion |
NveBeautyBlurTypeEnum.AdvancedBuffingSkin1 | Advanced microdermabrasion 1 |
NveBeautyBlurTypeEnum.AdvancedBuffingSkin2 | Advanced microdermabrasion 2 |
NveBeautyBlurTypeEnum.AdvancedBuffingSkin3 | Advanced microdermabrasion 3 |
Whitening type
Name | Use |
---|---|
NvBeautyWhiteningTypeEnum.WhiteningA | WhiteningA |
NvBeautyWhiteningTypeEnum.WhiteningB | WhiteningB |
Name | Use |
---|---|
NveShapeTypeEnum.FACE_WIDTH | Narrow face,[-1,1] |
NveShapeTypeEnum.FACE_LENGTH | Small face,[-1,1] |
NveShapeTypeEnum.FACE_SIZE | Face slimming,[-1,1] |
NveShapeTypeEnum.FORE_HEAD | Forehead,[-1,1] |
NveShapeTypeEnum.CHIN | Jaw,[-1,1] |
NveShapeTypeEnum.EYE_SIZE | Big eyes,[-1,1] |
NveShapeTypeEnum.EYE_ANGLE | Corner of eye,[-1,1] |
NveShapeTypeEnum.NOSE_WIDTH | Thin nose,[-1,1] |
NveShapeTypeEnum.NOSE_LENGTH | Proboscis,[-1,1] |
NveShapeTypeEnum.MOUTH_SIZE | Mouth type,[-1,1] |
NveShapeTypeEnum.MOUTH_CORNER | Corner of mouth,[-1,1] |
Name | Use |
---|---|
NveMicroShapeTypeEnum.HEAD_SIZE | Shrink head,[-1,1] |
NveMicroShapeTypeEnum.CHEEKBONE_WIDTH | Cheekbones wide,[-1,1] |
NveMicroShapeTypeEnum.JAW_WIDTH | Jaw width,[-1,1] |
NveMicroShapeTypeEnum.TEMPLE_WIDTH | Temple width,[-1,1] |
NveMicroShapeTypeEnum.NASOLABIAL_FOLDS | Nasal pattern,[0,1] |
NveMicroShapeTypeEnum.DARK_CIRCLES | Dark circles,[0,1] |
NveMicroShapeTypeEnum.BRIGHTEN_EYES | Eye-catching,[0,1] |
NveMicroShapeTypeEnum.WHITEN_TEETH | Beautiful teeth,[0,1] |
NveMicroShapeTypeEnum.EYE_ANGLE_DEGREE | Eye angle,[-1,1] |
NveMicroShapeTypeEnum.EYE_DISTANCE | Eyes apart,[-1,1] |
NveMicroShapeTypeEnum.EYE_HEIGHT | Eye height,[-1,1] |
NveMicroShapeTypeEnum.EYE_WIDTH | Eye width,[-1,1] |
NveMicroShapeTypeEnum.EYE_ARC | Eye curvature,[-1,1] |
NveMicroShapeTypeEnum.EYE_Y_OFFSET | Eyes up and down,[-1,1] |
NveMicroShapeTypeEnum.PHILTRUM | Among people,[-1,1] |
NveMicroShapeTypeEnum.NOSE_BRIDGE_WIDTH | Nose bridge width,[-1,1] |
NveMicroShapeTypeEnum.NOSE_HEAD | Nose,[-1,1] |
NveMicroShapeTypeEnum.EYEBROW_THICKNESS | Eyebrow thickness,[-1,1] |
NveMicroShapeTypeEnum.EYEBROW_ANGLE | Eyebrow angle,[-1,1] |
NveMicroShapeTypeEnum.EYEBROW_Y_OFFSET | Eyebrow up and down,[-1,1] |
NvMicroShapeTypeEnum.EYEBROW_X_OFFSET | Brow distance,[-1,1] |
Meishe SDK For Android: List of effects names (meishesdk.com)
The various parameters of beauty, beauty and micro-shaping are from the descriptions of AR Scene in the beauty document warehouse. For more advanced usage, see the link above.
For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData
Name | Use |
---|---|
NveMakeupTypeEnum.Lip | Lipstick |
NveMakeupTypeEnum.Eyebrow | Eyebrow |
NveMakeupTypeEnum.Eyeshadow | Eye shadow |
NveMakeupTypeEnum.Eyelash | Eyelash |
NveMakeupTypeEnum.Eyeliner | Informer |
NveMakeupTypeEnum.Blusher | Blush |
NveMakeupTypeEnum.Shadow | Contour |
NveMakeupTypeEnum.Brighten | Highlights |
NveMakeupTypeEnum.ComposeMakeup | Makeup |
For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/FilterJsonParseData
For JSON parsing, installation, and related information, please refer to the documentation: samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/NvAssetManager samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/AssetInfoData samples/android/NvEffectKitDemo/app/src/main/java/com/meicam/effectsdkdemo/data/StickerJsonParseData
Custom effects
NveEffectKit In addition to the above existing features, users can create their own Meisak-supported effects, using the middle layer to render. For example, the background segmentation function (need to initialize the face and segmentation model in advance) is implemented as follows
//Create background segmentationNvsRational aspectRatio = new NvsRational(9, 16);NvsVideoEffect mSegmentationEffect = NvsEffectSdkContext.getInstance().createVideoEffect("Segmentation Background Fill", aspectRatio);mSegmentationEffect.setIntVal("Stretch Mode", 2);mSegmentationEffect.setMenuVal("Segment Type", "Half Body");mSegmentationEffect.setColorVal("Background Color", new NvsColor(0, 1, 0, 1));//Add renderNveEffectKit.getInstance().addEffect(mSegmentationEffect);//Add render according to indexNveEffectKit.getInstance().insertEffect(mSegmentationEffect,0);//Remove created effectsNveEffectKit.getInstance().removeEffect(mSegmentationEffect);
Package installation
The above contents basically involve package installation, and installation instructions are available on the official website
Meishe SDK For Android: Material pack installation (meishesdk.com)
Can use has good tools (NvAssetPackageManagerUtil) sample code:
//packagePath This is a placeholder path; replace it with the actual resource package path.String packagePath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemesh";// Resource package license file path.String packageLicPath = "assets:/beauty/shape/shapePackage/63BD3F32-D01B-4755-92D5-0DE361E4045A.1.lic";// If the resource package file ends with .facemesh, the type is ASSET_PACKAGE_TYPE_FACE_MESH; if it is a warp package, the type is ASSET_PACKAGE_TYPE_WARP.//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.facemeshint packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_FACE_MESH;//63BD3F32-D01B-4755-92D5-0DE361E4045A.1.warp// int packageType = NvsAssetPackageManager.ASSET_PACKAGE_TYPE_WARP;//shapePackageIDString shapePackageID = NveAssetPackageManagerUtil.installFxPackage(packagePath, packageLicPath, packageType);The parameters that need to be paid attention to are:
1、packagePath– packagePath– The path to the installation package, which can be the assets path or the local absolute path
2、packageLicPath– Specifies the license file path of the installation package
3、packageType– Installation package type. For details, see:Meishe SDK For Android: com.meicam.sdk.NvsAssetPackageManager Class reference (meishesdk.com)
Render
1.RenderCore Correlation initialization
//Note: The SDK must be initialized in advanceNveEffectKit.getInstance().initialize();2.NveRenderInput Instructions
NveRenderInput mainly includes rendering configuration (NveRenderConfig), texture (NveTexture), Buffer (NveImageBuffer), image format (pixelFormat). The specific parameters are as follows:
(1)NveRenderConfig parameters, including camera orientation(cameraOrientation) and render mode (NveRenderMode)
NveRenderMode Instructions NveRenderConfig.NveRenderMode.TEXTURE_TEXTURE Texture in, texture out NveRenderConfig.NveRenderMode.BUFFER_TEXTURE Buffer in, texture out NveRenderConfig.NveRenderMode.BUFFER_BUFFER Buffer in, Buffer out (2)NveTexture Correlation parameter
Argument Instructions textureId Texture ID mirror Whether to flip horizontally, OES Texture to Texture 2D texture required NveSize Texture width and height NveTextureType Texture type NveTextureLayout Texture direction
NveTextureType Instructions NveTextureType.NV_COMMON_TEXTURE Ordinary texture NveTextureType.NV_OES_TEXTURE OpenGL OES texture
NveTextureLayout Instructions NveTextureLayout.UP_SIDE_DOWN Upside down NveTextureLayout.DOWN_SIDE_UP Normal (3)NveImageBuffer Correlation parameter
Name Instructions data Buffer data NveSize Buffer width and height mirror Whether horizontal flip displayRotation The Angle that the current image needs to be rotated rowPitchY The number of bytes per line of the image in the Frame, if the image format is NV21,YUV420, refers to the Y plane rowPitchUV The number of bytes per line of the image in the Frame is generally referred to as the UV plane, which is meaningless if the format is RGBA (4)pixelFormat Image format, which is encapsulated in NvFormatType. For details, see the official websiteVideo frame pixel format
3.NveRenderOutput Instructions
NveRenderOutput mainly includes the render result (NveRenderError), the output texture (NveTexture, parameters are the same as above), and the output Buffer (NveImageBuffer, parameters are the same as above). The render result parameters are as follows
Name Instructions NveRenderError.NO_ERROR Successful NveRenderError.ERROR_UNKNOWN Unknown error NveRenderError.INVALID_TEXTURE Invalid texture NveRenderError.INVALID_INPUT_PARAM Invalid input parameter for NvRenderInput 4.render
//Input configurationNveRenderInput mRenderInput = new NveRenderInput();mRenderInput.setPixelFormat(NveFormatType.VIDEO_FRAME_PIXEL_FROMAT_NV21.getType());NveRenderConfig config = new NveRenderConfig().setRenderMode(NveRenderConfig.NveRenderMode.TEXTURE_TEXTURE).setCameraOrientation(mCameraProxy.getOrientation()).build();mRenderInput.setConfig(config);//Input textureNveTexture nvTexture = new NveTexture();nvTexture.setTextureId(displayTex);Texture.setTextureType(NveTextureType.NV_OES_TEXTURE);nvTexture.setSize(new NveSize(texWidth, texHeight));nvTexture.setMirror(mCameraProxy.isFlipHorizontal());mRenderInput.setTexture(nvTexture);//Input BufferNveImageBufferbuffer = new NveImageBuffer();buffer.setData(data);buffer.setSize(new NveSize(bufferWidth, bufferHeight));buffer.setDisplayRotation(mOrientation);buffer.setMirror(mCameraProxy.isFlipHorizontal());mRenderInput.setImageBuffer(buffer);//renderNveRenderOutput renderOutput = mNvEffectKit.renderEffect(mRenderInput);......if (null != renderOutput) {//recycle render outNveEffectKitt.recycleOutput(renderOutput);}5.Resource release
When exiting or no longer need to render, you need to release the relevant resources, you can call the following method
NveEffectKit.getInstance().cleanUp();Note: This method needs to be called on the thread where GLContext is located
Destroy
To destroy the SDK in the case of page exit or app kill, call the following method
NveEffectKit.getInstance().destroy();Note: If you destroy a page when it exits, then you need to use the SDK elsewhere, so you need to re-initialize the SDK before using it, that is, call the init method
FAQ
1.When using a newer version of Android Studio, running the APK may fail. In this case, you need to change the Gradle JDK version to JDK 11.
>Settings->Build,Execution,Deployment->Build Tools->Gradle->Gradle JDK (
java 11
)Example of a similar error:
FAILURE: Build completed with 2 failures.1: Task failed with an exception.-----------* What went wrong:Execution failed for task ':app:processDebugMainManifest'.> Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @16a1c82d* Try:Run with --info or --debug option to get more log output. Run with --scan to get full insights.* Exception is:org.gradle.api.tasks.TaskExecutionException: Execution failed for taskCaused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.String java.io.File.path accessible: module java.base does not "opens java.io" to unnamed module @16a1c82d2.What is the difference between face240 and face106 in the model?
240 and 106 refer to the number of facial landmark points; 240 provides better results.
Makeup is not supported with 160-point landmarks.
3.What is the difference between Standard Skin Smoothing, Advanced Skin Smoothing 1, 2, and 3?
Advanced Skin Smoothing
1
is the standard effect forAndroid
, Advanced Skin Smoothing2
is the standard effect foriOS
, and Advanced Skin Smoothing3
is optimized formale
subjects.See documentation:AR Scene Advanced Beauty Type.
4.What is the difference between Beauty A and Beauty B?
They use different implementation methods;
Beauty B
uses aLUT
file. See documentation:AR Scene Whitening Lut File.5. Is custom GLSurfaceView/camera preview supported?
Sample code is provided for reference. If you already have your own implementation, you can provide the texture or buffer data to be rendered. For more details, please contact Meishe Technical Support.
6.No rendering effect or black screen after importing the project?
Check whether the
lic
file in your project is correctly bound to yourapplicationId
. For more details, please contact Meishe Sales.7.How to resolve conflicts after integrating NvAndroidStreamingSdk.jar?
Remove the existing
NvAndroidStreamingSdk.jar
from your project and useNveEffectKit.aar
instead.