How To Use Animated Map For Facial Expressions

Animating expressive facial animation is a very challenging topic within the graphics community. In this paper, we innovate a novel ERI (expression ratio image) driving framework based on SVR and MPEG-4 for automatic 3D facial expression blitheness. Through using the method of back up vector regression (SVR), the framework can learn and forecast the regression human relationship between the facial animation parameters (FAPs) and the parameters of expression ratio image. Firstly, we build a 3D confront animation system driven past FAP. Secondly, through using the method of principle component assay (PCA), we generate the parameter sets of eigen-ERI space, which will rebuild reasonable expression ratio image. And so we acquire a model with the support vector regression mapping, and facial animation parameters can be synthesized quickly with the parameters of eigen-ERI. Finally, we implement our 3D face blitheness organization driving past the consequence of FAP and information technology works effectively.

1. Introduction

Facial animation is one culling for enabling natural human-computer interaction. Reckoner facial blitheness has applications in many fields. For example, in the amusement manufacture, realistic virtual humans with facial expressions are increasingly used. In communication applications, interactive talking faces not merely make the interaction between users and machines more fun, but also provide a friendly interface and assist to concenter users [1, 2]. Among the issues concerning the realism of synthesized facial blitheness, humanlike expression is critical. Only how to analyze and comprehend humanlike expression is still a very challenging topic for the computer graphics community. Facial expression analysis and synthesis are an active and challenging enquiry topic in computer vision, impacting important applications in areas such as human-figurer interaction and information-driven animation. We introduce a novel MPEG-iv based 3D facial animation framework and the animation system driving past FAP that produced from camera videos. The MPEG-4 based framework has the advantages of currency and few information [3–five]. Get-go the system takes 2d video input and recognizes the face area. Then the face prototype was transformed to ERI [vi]. Next, a simple ERI'south parameterized method is adopted for generating an FAP driving model by the back up vector regression and it is a statistic model based on MPEG-4. The results of FAPs tin can be used to drive the 3D face model defined past MPEG-4 standard.

The remainder of the paper is organized as follows. We present the related piece of work in Section 2. We then describe how to preprocess video information in Section 3. Section 4 presents how to construct the eigen-ERI. Section 5 describes the extraction of the FAP. In Section 6 we propose a novel SVR-based FAP driving model. Finally, we testify the experiments and conclude the paper in Sections 7 and 8.

Recently, realistic facial animation has become i of the about important research topics of figurer graphics. Many researchers focus on the nature of the facial expression. In [seven, 8], a sample-based method is used to make photorealistic expression in item. In particular, Chang et al. [9] implement a novel framework for automatic 3D facial expression analysis in video. Liu et al. proposed an ERI-based method to extract the texture depth information of the photograph in [6], and Effigy 2 shows the ERI of the frown expression. For reflecting the changes of a face surface, Tu et al. [10] utilise the gradients of the ratio value at each pixel in ratio images and apply information technology to 3D model. In addition, a parameterized ERI method was proposed by Tu et al. [ten] and Jiang et al. [11].

Zhu et al. [12] used a method of SVR-based facial texture driving for realistic expression synthesis. In Zhu's piece of work, a regression model was learned between ERI'due south parameters and FAPs. According to the inputs of FAPs, the model will forecast the parameters of ERIs. And so a reasonable facial expression image was generated with ERIs. On the contrary, our method is to forecast FAPs from the parameters of the eigen-ERIs; furthermore we realize a 3D confront animation system driving past FAPs.

In this paper, we realize a 3D face blitheness organisation that can generate realistic facial animation with realistic expression details and can apply in dissimilar 3D model similar with homo. The main problems of our facial animation system are the extraction of the ERI from the camera video and learning the SVR-based model, furthermore building an FAP driving 3D facial expression blitheness organization. Next section will introduce these.

3. Video Data Preprocessing

iii.1. Video Data Capture

For the robustness of the algorithm, we go video data in normal lite surroundings, and video equipment is often used to support continuous capture PC digital photographic camera. Then we set the sampling rate of 24 frames per 2d and the sampling resolution of 320∗240, and the total thou sampling frame was captured, of which 200 expressions defined primal frame.

In this newspaper, we adopted the method of the marked points in face to extract facial motion data, and the virtually difference with other methods is that our method is based on the MPEG-4 standard (Figure 1(a)). The reward of the standard is that you can share daters, and the data tin be used with whatever standards-based grid. By marked points, we can become more accurate facial movement data, while blue circle chip can be pasted in any place, including eyebrows and eyelids, and have no effects on their movement, which makes it possible to get the whole facial move data and satisfy with the processing of the side by side step.

(a) AAM

(b) Mesh triangle

In order to obtain experimental data, first nosotros make a fibroid positioning through the blue calibration point using face detection tools and accept calibration and reduction transformation operations on the sample data. Then nosotros blueprint a face mesh model in Effigy ane to describe the geometry of the facial animation. Mesh vertices were mapped automatically using agile advent model (AAM) and manual fine tuning to meet subsequent need for data extraction.

After obtaining the data of the texture and characteristic points, all texture volition exist aligned to the average model.

3.ii. Features Points

According to MPEG-4, nosotros divers six basic expressions types (see Effigy 3) and captured them from video camera. Twenty-four key frames demanded for training and one nonkey frame used for testing per expression type are extracted with resolution of 320∗240 pixels. In Figure 1(a), the 68 feature points are marked automatically with AAM [xiii]. In our experiment, nosotros adopted the 32 feature points belonging to the FPs in MPEG-4.

4. Computation of Eigen-ERI

4.1. The Computation of the ERI

For edifice general ERI'southward model, we used a large of frontal and neutral expression face as sample library. And nosotros excerpt the outline of the face features by the method of the active appearance model (AAM). Then the following formula was defined to compute the boilerplate face's ERI, as the standard shape model:

Here means boilerplate face, and ways any 1 face up , and means the number of the frontal and neutral expression faces.

For getting facial expression details, nosotros compute the ERI from all key frames sequences continuously. The first frame is denoted past and regarded as a neutral face image. denoted the residual of expressive face up images samples, where n is the full number of key frames. Each expressive face sample volition be aligned to . Then, the ERI of the sample sequence can be computed as follows:

Here, the  announce the coordinates of a pixel in the image, and and denote the color values of the pixels, respectively.

Considering the computation of ERI is each point one by ane according to [14], the results will exist unpredictable if the face features cannot be aligned exactly. So we revise the definition of ERI'southward computation

iv.two. Eigen-ERI

The matrix of ERI is the second grayness image . Nosotros correspond it with the dimension vector . The training sets are ; the means the sum of the images. The average vector of the all images is

The difference value of the ERI'due south and the boilerplate of the ERI'southward were divers as :

The covariance matrix is represented as follows: where .

Feature face is composed of the orthogonal eigenvector of the covariance matrix.

ERI calculation only takes the eigenvector corresponding to the largest eigenvalue. The is decided by the threshold of the : In this paper, nosotros select the maximal 21 variables in the eigen-ERI space to correspond 96% variation information in the sample sets.

v.1. Definition of the FAP (Facial Animation Parameter)

The FAP is defined past a set of facial animation parameters in MPEG-iv standard. FAP based on the face of small deportment is very shut to the facial musculus movement. In fact, FAP parameter represents a set of basic facial movements, including the head movement control, tongue, eyes and lips, facial expression and lip tin reproduce the most natural motion. In addition, like those of humans did not exaggerated facial expressions cartoon, FAP parameter can exist traced.

At that place are six bones expressions types, defined in MPEG-4 (see Figure three). The MPEG-iv has a total of 68 FAP. The value of FAP is based on FAPU (facial animation parameter unit) as the unit, so that FAP has the versatility. The calculation conception is

Among them,   and  are neutral, facial feature points on the corresponding parts.

5.2. The Implement of Face Animation Based on FAP

Facial animation definitions table defines three parts. First, the FAP range is divided into several segments. Second, nosotros want to know which grid points are controlled by the FAP in the mesh. Third, nosotros must know the motion gene of the control points in each segment. Each FAP needs to detect out which parts of the three parts in facial animation definition table, and then co-ordinate to the MPEG-4 algorithm, calculated by the displacement of the FAP command all grid points. For a set of FAPs, each FAP calculated the constructive grid point rod size; by the shift-and-add up yous will get a bright facial expression (Figure 3). The concrete implement may refer to [15].

6. SVR-Based FAP Driving Model

vi.ane. Support Vector Regression (SVR)

In a typical regression problem, we are given a training set up of independent and identically distributed examples in the form of ordered pairs, , where and denote the input and output, respectively, of the thursday training case. Linear regression is the simplest method to solve the regression trouble where the regression role is a linear role of the input. As a nonlinear extension, back up vector regression is a kernel method that extends linear regression to nonlinear regression by exploiting the kernel trick [xvi, 17]. Substantially, each input is mapped implicitly via a nonlinear regression map to some kernel-induced feature space where linear regression is performed. Specifically, SVR learns the following regression function by estimating and from the training data: where denotes the inner product in . The trouble is solved by minimizing some empirical risk mensurate that is regularized appropriately to control the model capacity.

Ane unremarkably used SVR model is chosen -SVR model, and the -insensitive loss office is used to define an empirical take a chance functional, which exhibits the same sparseness property as that for back up vector classifiers (SVC) using the swivel loss office via the and then-called back up vectors. If a information indicate lies within the insensitive zone called the -tube, that is, , so information technology will not incur any loss. However, the error parameter has to be specified a priori past the user. The primal optimization trouble for -SVR can be stated as follows:

Relatively, (3) transforms into

Here, denotes Lagrange multiplier, is kernel role, and is decision function. Forpredicting, it is but a dot production operation and costs very low in the existent time. Effigy 4 shows the steps of the SVRM mapping ERI to FAP parameters.

6.2. SVR-Based Algorithm

Co-ordinate to the above theory, an FAP driving model for its every parameter is congenital. Given a set of training data , where denotes the space of the ERI and is the ERI's parameter and denotes the space of the characteristic of FAP and is an FAP. We regard it as a regressive problem and represent it by the support vector regression method, including -SVR [16] and -SVR [17]. The offline learning progress is equally Figure 5.

An offline learning process was showed in Figure 5. In the above section, we accept got the learning parameters of the FAPs and ERIs. So we tin can get a regressive model from the ERI's parameters to the FAPs vectors through the SVR model. In a statistical sense, the results of FAPs can reflect the expression relative with the ERI.

There are several steps used to explain how the facial expression animation is driven past video photographic camera.

Step ane. Constitute the MPEG-four based 3D facial expression animation system driving past FAP; run into Effigy 6.

(a) 3D mesh

(b) Synthesis confront

Step two. Capture and notice confront image from video camera, and compute its ERI.

Pace 3. Forecast the FAPs of the current frame according to its ERI.

Step 4. Compute the motion of the characteristic points in the face mesh based on the new FAP and breathing the 3D human confront mesh.

7. Experiment Results

We have implemented all the techniques described higher up and built an automatic 3D facial expression animation arrangement on Windows environment. The event was showed in Figure 7. Nosotros correspond half dozen bones expressions driving by FAP which come from the forecast through ERI.

(a) Anger

(b) Joy

(c) Disgust

(d) Sadness

(e) Surprise

eight. Conclusion

In this paper, nosotros realize a 3D face blitheness system that can generate realistic facial animation with realistic expression details and can apply in different 3D model similar to human being. It is capable of generating the statistical realistic facial expression animation while just requiring but camera device as the input data and it works better in any desired 3D face model based on MPEG-4 standard.

Conflict of Interests

The authors declare that there is no disharmonize of interests regarding the publication of this newspaper.

Acknowledgment

This research was supported partly by NSFC Grant no. 11071279.

Copyright

Copyright © 2014 Shuo Sun and Chunbao Ge. This is an open access article distributed under the Creative Eatables Attribution License, which permits unrestricted utilise, distribution, and reproduction in any medium, provided the original work is properly cited.

Source: https://www.hindawi.com/journals/jam/2014/706159/

Posted by: reagancitaks.blogspot.com

Share this post