Skip to main content

Attribute Search and People Search with Faces

  • Updated
 

Searching footage for a specific person or vehicle can sometimes be a difficult task. However, with Spot’s new features: Attribute Search and People Search with Faces the process has never been easier. You can now search video for clothing (upper or lower body), vehicle color, and vehicle make and type.

The first step to using either search feature is to enable it through your Spot Dashboard. In the Dashboard, navigate to Configure > Settings. At the bottom of the list, you’ll see People and Vehicle Attribute Search, and People Search with Faces. Enable just People and Vehicle Attribute Search if you don’t intend to use People Search, or enable both if you do. This toggle will enable all cameras across the organization that are capable of these search options.

For information on how to bulk edit individual cameras for Attribute Search and People Search with Faces, please see this article.

Attribute Search

Step 1: Navigate to the Search tab in your Spot Dashboard, then select the Cameras tab.

Step 2: Select a camera with attribute search enabled. 

Step 3: Select People or Vehicles to the right of the date and time selections.

Step 4: Select the attributes you’d like to search on. 

  • People Attributes include Upper Clothing Color and Lower Clothing Color
  • Vehicle Attributes include Vehicle Color, Vehicle Make, and Vehicle Type

Rule of thumb: Enable filters one by one until you find the desired clip. Inaccuracies from each filter will compound which could lead to no clips being found if the results are over-constrained. Best filters are ordered left to right. 

  • Multiple attribute selection is supported (black OR white, OR gray)
  • Multiple filter selection is supported (black upper clothing AND gray lower clothing)

Note: The Dashboard will automatically disable any filters that are incompatible with the camera that you are searching.

Step 5: Scrub and Search Video

  • Clips will appear based on the attributes searched, including the filters used. You can remove filters by clicking the “X” to expand the number of segments shown.

For information on how to scrub and search by segments, please see this article.

People Search with Faces

Note: The first time People Search with Faces is used, you will be presented with a clickthrough screen. These are the legal terms and conditions that must be abided by to use this feature.

Additionally, People Search depends on your feedback and coaching to give best results. The more you coach People Search, the smarter it gets. Please see the section below on coaching to read more. 

Step 1: Navigate to the Search tab in your Spot Dashboard, then select the People tab.

Step 2: Select the filters you’d like to search by from the selections on the left side of the screen.

  • For time/location/camera filters, only the faces seen at those times/in those locations/on those cameras will be available to search. 
  • For attribute features, only the faces that have matching attributes will be available to search.
    • Attributes include Upper Clothing Color and Lower Clothing Color 

Step 3: Select a person from the results list to see all video clips associated with that person.

The appearances of the chosen person are seen below the key image. It shows the first and last date of sighting, locations, and camera names. You can change the desired filters with the blue "Filters" button on the top right-hand corner of the search page.

Step 3: Sort and use detected clips.

You can sort the clips by time, location, or cameras. This will update the clip headers in the center of the view. Once you find the clip you need, you can share it, save it, add it to a case, or download it - whatever is necessary to complete your investigation. 

Metadata on People

Renaming People

You can name, or rename, a subject once they're located. To do this, click the pencil icon to the right of the existing name, type in the new name, and click the check to save.

Deleting People

If you need to delete a person for any reason, or, if the results return something that is not actually a person, a subject can be deleted in full. This action is irreversible. To do this, simply click the trash icon to the right of the name, and click delete on the prompt box that appears. 

People Search Coaching

At SpotAI we want to give customers the best tools to solve their use-cases with video. However, depending on your camera set up (angle, location, lighting, etc) you may see different results. Some things you may see are: 

  • Multiple faces of the same person may be split into multiple subjects
  • Faces may be assigned to the wrong person

People search depends on your coaching to provide you with the best results. People Search with Faces gives users the ability to coach and correct the system with the following options: 

  • Merging people
  • Rating clips within a person's clip view

When coaching People Search, Spot learns from the correction by assigning future faces to the correct person. This is an instantaneous change that happens after every rate or click. 

Merging Profiles

Step 1: Access People Search

Step 2: Select "Combine & Improve" to activate merging mode

Step 3: Select all the people you would like to combine then click the "Combine x People" button

Step 4: If the merge is successful, you'll get a notification saying people have been combined, with a link to view the newly combined person. Click to see the updated person combinations. 

Rating Clips Within a Profile

To help increase the accuracy of each person's profile, it is encouraged to rate the found clips. Doing so is simple and the results are automatic. 

Step 1: Open a person's profile

Step 2: Hover over a clip, if it is OK press the thumbs up button and stop here. Spot will learn from this rating. If it is Not OK press the thumbs down button and proceed to the next step.

Note: Once you have rated a clip, you cannot undo the rating. If you find that you have mis-rated someone, you must reassign the clip to the correct person.

Step 3: If the clip is Not OK, a pop up will appear with three options: 

  • Create a New Person
  • Reassign to Existing Person
  • Not a Person

Create a New Person

If the clip contains a new person, choose Create a New Person. The clip will be immediately reassigned to a new person, and you'll get a confirmation message. You can click the provided link to immediately view this new person. 

Reassign to Existing Person

If the clip contains an existing person, choose Reassign to Existing Person. You'll then be prompted to choose the correct person from the list of Existing People.

If the assignment is successful, you'll get a confirmation message. You can click the provided link to immediately view this person. 

Not a Person

If the clip is not a person, choose Not a Person. This will immediately delete the clip from the person subject view, however the video itself is still retained. 

Technical Requirements

These features will work on any appliance starting with sn6g or higher, running firmware version 4.3.0 or higher, as long as the features have been turned on through the Configure tab.

Camera Setup

Once the features are enabled, you can edit your installations and camera settings to get the best possible results for your location and use-case. Please use these guidelines to achieve the best accuracy and results. 

Cameras and Camera Settings

Spot cameras do not need adjusting. If however you are using cameras pre-installed at your locations, or want to buy your own cameras, take the following into consideration:

  • At least 1080p Resolution: Opt for a camera with a minimum of 1080p resolution. 5mp resolutions provide clearer details and improve facial recognition's accuracy, however they may impact the available storage on your IVR. 
  • Fixed lenses preferred: Opt for fixed lenses for best quality and setup maintainability. Zoom lenses can be considered if you have a special use case, however zooming in typically causes more movement and blurring that can adversely affect the feature.
  • Enable Fast shutter speed: Opt for higher shutter speed that reduces motion blur. This may cause your image to be darker, so please refer to suggestions on lighting below. Motion blur heavily impacts accuracy for people search.
  • Enable Auto White balance: This enhances detection of true shirt colors.
  • Prefer Visible Light Cameras, and add lighting if necessary: IR is suitable for low-light conditions, but grayscale images won't support shirt color detection. Visible light cameras are crucial for color detection. Additionally, grayscale reduces information possible for faces, which may cause those faces to be incorrectly classified.
  • Limit noise: low lighting or bad sensors may introduce noise, which adversely affect detections quality.

Camera Placement

  • Avoid obstructions: Prevent obstructions like pillars or decorations from blocking the camera's line of sight.
  • Do not install cameras upside down: If it is upside down, correct the view in camera settings.
  • People search:
    • Distance: < 10 feet desired from face. <20 feet ok from face. A face should take up a 100x100 pixel area on the screen. If this is not possible, then a 50x50 square can be classified — however, this may lead to suboptimal accuracy.
    • Angle: Prefer eye level, but install at a height of less than 8 feet, less than 20deg left and right of face: Install cameras at face level to minimize distortions.
  • Basic Attributes: Colors
    • Any distance of which a person is detectable on the spot dashboard.
  • Advanced Attributes (Vehicle Type and Make)
    • For optimal results:
      • Distance: < 30 feet from camera
      • Angle: 
        • Front or back detection: less than 40 deg from the front or the back of the car.
        • Side detection: Generally, recognition from the side is more difficult because “key features” are not visible. However, also prefer less than 40 degrees of the car.
      • “Key features” of the car, such as the logo, or the “F150” sign on a ford, etc. should not be occluded.

Lighting Considerations

  • Consistent: Aim for consistent illumination to prevent subjects from transitioning between overly bright and dark areas.
  • White: Ensure lighting sources have uniform white color temperatures.
  • Artificial lighting: For nighttime or low-light conditions, consider motion-activated lights.

Post-Installation Adjustments

  • Testing and Refinement: Test the setup under varied conditions. Use subjects with diverse skin tones, clothing, and heights to validate accuracy. Adjust settings or positioning as necessary.
  • Camera Hygiene: Ensure lenses are clean, and periodically recalibrate focus, exposure, and white balance.

Limitations and Legal Requirements

While color attributes (upper and lower clothing and vehicle color) will be available for both Core and Pro licenses, any non-color attributes (vehicle make and vehicle type) will only be available on Pro licenses. More information on the licensing packages is available here.

Each user will need to agree to the following clickthrough agreement to proceed: 

The Spot AI Dashboard offers a people search tool that requires the use of facial recognition technology and biometric data (“People Search Tool”). Due to numerous state and local laws governing the use, storage and processing of biometric data, you must carefully review and understand the following terms that govern the People Search Tool prior to your use.

In using the People Search Tool, you acknowledge and agree that:

  1. you will not use the People Search Tool, including turning on the People Search Tool in the following states:
    1. California
    2. Colorado
    3. Florida
    4. Illinois
    5. Texas;
  2. you will not use the People Search Tool, including turning on the People Search Tool in the city of Portland, Oregon;
  3. certain cities have banned government use of facial recognition technology and you will not use the People Search Tool if the ban in those cities applies to your situation;
  4. you will only use the People Search Tool for the following limited purposes:
    1. detecting, preventing, protecting against, or responding to a security incident, identity theft, fraud, harassment, malicious or deceptive activity, or any illegal activity
    2. investigating, reporting, or prosecuting a person responsible for an action described in section (4)(a)
    3. preserving the integrity or security of systems
    4. in the State of Utah only, for the additional limited purpose of investigating, reporting, or prosecuting a person responsible for harming or threatening the integrity or security of systems;
  5. we reserve the right to restrict, suspend or disable your use of the People Search Tool if we believe you are violating these terms or the Spot Terms of Use;
  6. your use of the People Search Tool outside of the geographical scope in sections 1 and 2 and outside the scope of use in sections 3 and 4 are material violations of the Spot AI Terms of Use and may result in suspension or termination of the services including the People Search Tool;
  7. your use of the People Search Tool:
    1. is reasonably necessary and proportionate to the purposes listed in section 4 above
    2. is adequate, relevant, and limited to what is necessary in relation to the specific purposes listed in section 4
    3. includes your consideration of the nature and purpose of the collection, use, or retention of the biometric data collected, used, or retained;
  8. the biometric data must be subject to reasonable administrative, technical, and physical measures to protect the confidentiality, integrity, and accessibility of the biometric data and to reduce reasonably foreseeable risks of harm to consumers relating to the collection, use, or retention of personal data;
  9. the requirements set forth in these terms are not intended to, and shall not, be construed as legal advice, and you should seek the advice of counsel, as necessary, to understand, interpret and comply with the various state and local laws that may be applicable to your specific situation; and
  10. the laws governing the use of facial recognition technology and biometric data are continuing to evolve and we reserve the right to update these terms accordingly, in which case you will be notified of such updated terms via a new click-through agreement.

 

BY CLICKING THE “ACCEPT” BUTTON OR CHECK BOX PRESENTED WITH THESE TERMS, (A) YOU ACKNOWLEDGE THAT YOU HAVE READ, UNDERSTOOD, AND AGREE TO ABIDE BY THE TERMS; AND (B) YOU HAVE RECEIVED PERMISSION FROM AN AUTHORIZED REPRESENTATIVE OF YOUR EMPLOYER TO USE THE PEOPLE SEARCH TOOL IN ACCORDANCE WITH THESE TERMS. IF YOU DO NOT AGREE TO THESE TERMS OR YOUR EMPLOYER DOES NOT GIVE YOU PERMISSION, YOU MUST NOT USE THE PEOPLE SEARCH TOOL.

Accuracy and Environmental Considerations

AI Search is a constantly growing and changing field, so there are a few things to consider when searching by color or people attributes. The closer a subject is to the camera, the more accurate the search will be. Please refer to the Camera Setup Guide for concrete details, however here are a few tips you can consider for your camera set up to take the most advantage of Spot AI:

People Search: 

  • The best search results come from cameras that are 20 feet or less away from the subject, and when the subject is looking straight at the camera.
  • While rating and retraining can help increase accuracy, repositioning cameras may result in a better outcome overall.

Vehicle Search:

  • The best results come from cameras up to 30 feet away. 

Color accuracy:

  • Too dark: If the lighting is too dark, for example, the color may default to black.
  • Occlusions: If a customer is standing in front of a blue counter, their lower clothing color may be detected as blue.
  • Color bleed: If there are colored or decorative lights, those colors may blend with the actual color of the item resulting in a different color being detected.

In order for the system to generate search results, still frames are downloaded to the cloud. These downloads, depending on location activity, can greatly increase bandwidth consumption. For cameras with attribute and people search enabled, it is expected to see an average increase in bandwidth of up to 100kbps per camera (depending on the motion seen in the camera). If the bandwidth usage is too high, you can individually disable Attribute Search and People Search via the Maintain tab. Please see this article for that process. If there is a network issue, or other issue that prevents the frames from being downloaded to the cloud, both Attribute Search and People Search will not work. 

  

We hope this article was useful to you, please leave us a comment or feedback as it will help us improve our customer support center.

Related articles

Comments

0 comments

Please sign in to leave a comment.