is a public and paid announcement of a persuasive
message by the company. Nowadays,
signage had been changing from
traditional printed signage to digital signage. The purpose of digital signage
is to deliver the message and information to an audience
which includes news, notice, movie
trailer and advertisement. Digital signage can be easily implemented by using
liquid crystal display (LCD) or projection to deliver the message.
Digital signage can be easily found almost everywhere such as
shopping malls, airports, restaurants, and station. Digital signage has more
advantages compared with traditional
signage because it can display information and advertisement not only in image.
It can deliver the message and information using multimedia content as video,
animation, and audio. Some digital signage also requires
the interaction between the audience and the advertisement.
Targeted advertising is an advertisement that served as audience
demographic for the advertiser to know what the target audience. To establish
the targeted advertising with digital signage, audience biometrics characteristic
such as the face, iris and gait are
required for apply biometric detection. In this research, we are using
multimodal biometric that combine several types of the biometric characteristic for detection. The attributes features that obtain from the
multimodal biometric such as gender, age, glasses, and beard will be used as a statistic for a broadcasting
selected advertisement for the targeted audience according to audience
Face recognition is a computer application that identifying or
verify a person from image and video.
Face recognition play an important role in
targeted advertising because it can detect the face multimodal such as age,
gender, beard and skin color. With the
processing of the face recognition, it can develop adaptive appearance model
for monitoring the audience demographic.
In order to
recognize the audience, face consists the crucial information for
identification. But most of the face detection and face recognition able to
detect the frontal view or side view of the face. The result will be affected if putting the side view of the
face into the face detection. To avoid this problem, it only requires the frontal view of the face of the audience to classify as the viewers
who look at the digital signage and the side view as the passerby who does not look at the digital
Since the real-time environment is unconstrained, the
image that captured by the camera may affect by
the light of the environment. It will let the image provide inaccurate
information and lead the process to misclassification. So, classifying based on
multiple images is much more reliable. Since the thesis focusing on the public area, if multiple audiences are
standing in front of the digital signage,
the face of some audience may be messing up or cover by other audience who
stand in front of him.
earlier, this thesis focusing on pubic audience demographic, the faces that
capture by the camera may possible to be small due to the distance between the camera and the audience. The low-resolution image will get the smaller image
input. So, the smaller the input of the image, the shorter time of the
processing is required. In order to minimize the delay of the advertisement in
digital signage, a good conduct will keep
the faces resolution as low as possible and the classification of the image
will keep process for the next advertisement during the audience viewing the
1.3 Research Objectives
The main objective of the thesis is to create a
prototype to simulate the performance of a multimodal identification for a
group of audience in the camera view. In order to achieve it, the following
objectives are outlined:
To design a new framework for
targeted advertising in public display
To develop an adaptive
appearance model for monitoring the audience demographic
To define a representation set
of measurement for investigating the performance of targeted advertising.
Real-time audience demographic application was
developed for digital signage that combines
the LCD screen and digital camera. The digital camera will capture the video of the audience who stand in
front of the digital signage. By using the computer vision method, the
application performs face detection and
tracking method to detect audience face image who stand inside region of
interest and viewing on the screen. After the audience face was detected, the
application will perform several classifications
for gender, age, glasses and beard detection. The database will include the
time of the advertisement, gender, age, beard and whether the audience wearing
the glasses or no. The result will be saved in the database as audience demographic. Besides that, the application
also will perform decision about broadcasting what category of an advertisement according to audience
Digital signage is now
gaining popularity and ubiquitous on advertising. It can display image,
message, animation, video, and text.
Digital signage consists display devices as liquid crystal display and display
controller as PC or digital media player. It can be found in public and high population
areas such as shopping mall, retail shop,
restaurant, bus station and corporate building. Digital signage has become an
important platform for delivery
information, message, and announcement to
the public. Figure 2.1 shows the simple
deployment of digital signage.
Most of the organization
has changed from conventional signage to
digital signage. Conventional signage is a traditional advertising method that delivers information at a specific location. Conventional signage has
many disadvantages on time-consuming and
cost of the construction and installation of the signage. The other
disadvantages of conventional signage are
it only can display static information such as image and text. Digital signage
reduces the cost of construction and installation of the conventional signage.
It able to display image, animation, video, text and interactive interfaces. Moreover,
it also supports live video and web
With the evolution of the
technology, some digital signage had come with touch screen, body sensor or QR
code via smartphone. It allows the
audience to interact with the digital signage system. It will increase
audience’s user experience of interacting with the advertisement of the digital
signage (Using Interactive
Digital Signage to Increase Customer Engagement, 2017). Networked digital signage is controlled via a computer networks such as internet or intranet
and connects to the digital display. Figure 2.2 shows the networked digital signage.
Targeted advertising is an advertisement that used to serve a specific
audience which can be a group of people, individual or a particular
audience demographic. Instead of promoting
advertisement to the public, targeted
advertising allows the advertisement to reach to the right people as
many as possible. Digital signage also applies
the targeted advertising to increase the effectiveness of the digital signage
content (Why is digital signage effective?, n.d.). This enables the digital signage content able to
deliver a message to the right audience
at any time.
Targeted Advertising in Online
Online advertising also knows as online marketing. It is a marketing
plan and advertising which allow the user to promote their product or service
to consumers through the internet.
Facebook and Google were the examples of
the online advertising that was using
targeted advertising. With using targeted advertising, Facebook able to let the
advertiser promote their product or service to the targeted consumer based on
the location, age, gender or keyword that consumer search on Facebook.
There are three type of
targeting advertising methods:
Behavioural Targeting – The
advertiser uses the activity and action of the user such as the pages that they
visit or search to determine which advertisement to show.
Content and Contextual
Targeting – The advertiser targets people based on
the content people are reading on the website
for keyword and show advertisements to the webpage based on the keyword.
Geographic targeting – The
advertiser target user based on the geographic location. Internet protocol
address (IP address) can be signal the location of the user. After locating the location of the user, it will
display the advertisement based on their geographic location.
Targeted Advertising in Digital
To establish targeted advertising on digital signage, the human face is required for face detection and
face recognition. During the advertisement played, the camera was used to capture the audience’s face image who look at
the digital signage. The system will detect the audience’s face using Haar
cascade and pass the face image into the framework
to extract the demographic data. After that, the result of the data will be used to find out a right advertisement to play to the audience.
Figure 2.3 shows the flow of the targeted
advertising in digital signage.
Figure 2.3: The
basic flow of targeted advertising in digital
The human face is a source of information about the attributes of a
person such as age, gender beard, skin color,
and glasses. Face detection can be used to identify the face multimodal
attributes and develop an adaptive
appearance model for monitoring the audience demographic. 2 It enables digital signage to increase the
effectiveness of targeted advertisement played in
The process of extracting the human face
image from image or video from the real-time application. Face detection finds
out the location and the size of the face image by using Haar Cascade feature.
Haar cascade feature is an object detection method that proposed by Paul Viola
and Michael Jones in 2001. 1 Nowadays, this feature method still used widely
in face detection research. It is a machine learning method that trains the
image inside the database and used to find out and locate the object in another
image. The images that used for trained will be separated into two types:
positive and negative images. For face detection, positive images are the image
that includes the human face and negative image is the image that not include
any human face image. After training the positive and negative images, the
machine learning will update the classifier and the classifier are able to find
out and detect the face image in another image by extracting the feature. If the
performance is weak, that is the high false positive and false negative rates.
After the human face was detected, we want
to track the face of the person with using Continuously Adaptive Mean Shift (CamShift)
method. 4 CamShift is the method that uses for object tracking and tracks the
color object with the combines of the mean shift. There is the problem of the
CamShift is when there has the same color object that covers the color image
that was tracking by the Camshaft. Face area can change according to the color
of the image that track.
The step of the Camshift algorithm: 4
region of interest (ROI) of the image.
the initial location of the search window with Mean Shift. Example: face image
that detects by Haar Cascade
the color probability of the center of the ROI.
shift to find out the centroid of the image and store it as the 0th movement.
next frame, repeat step 3 and step 4 until the mean shift location moving threshold
was less than a preset threshold.
Local Binary Pattern
Binary Pattern (LBP) is an approach method used for texture and image analysis.
Local Binary Pattern was first described by T.Ojala in 1994. 3 It has been
use as the powerful feature for texture
classification. Until now, it not only a method that used for texture
classification and it also becomes a method that used for face detection and