# CS代考 Computer Vision (7CCSMCVI / 6CCS3COV) – cscodehelp代写

Computer Vision (7CCSMCVI / 6CCS3COV)

Recap

• Image formation

● Low-level vision ● Mid-level vision

• •

grouping and segmentation (finding matching elements within an image)

correspondence problem (finding matching elements across images)

● for all locations or selected interest points

● comparing image intensities or descriptors

● finding matches by search

● determining similarity between elements

● dealing with false matches by model fitting

Stereo and Depth ←Today ● High-level vision

●

Computer Vision / Mid-Level Vision / Stereo and Depth

1

Today

• Stereo vision

– stereo camera geometry

» coplanar cameras (simple case)

» non-coplanar cameras (complex case)

– disparity measurement » calculating depth

– correspondence

» stereo constraints used to solve the correspondence problem

• Other cues to depth – Binocular

– Oculomotor – Monocular – Motion

Computer Vision / Mid-Level Vision / Stereo and Depth 2

Why is stereo vision important?

x1

2

z1

2

x ‘= f ‘ X 1= f ‘ X 2 Z1 Z2

A camera projects 3D points onto a 2D plane

3D points on the same line-of-sight have the same 2D image location (i.e. imaging results in depth information loss)

Computer Vision / Mid-Level Vision / Stereo and Depth

3

Why is stereo vision important?

Depth information can be recovered using two images and a knowledge of geometry.

e.g. all points P, P1, P2, and P3 project to the same location in the left image, but to different locations in the right image.

The right image allows us to measure how far each of these points are from the left camera (if we can solve the correspondence problem).

Computer Vision / Mid-Level Vision / Stereo and Depth 4

Why is stereo vision important?

This is useful for:

Path planning / collision avoidance (car / robot) virtual advertising

Not stereo, but same methods can be used for 3D model building

++

→

Computer Vision / Mid-Level Vision / Stereo and Depth

5

Stereo: coplanar cameras

OL

i

j

k

(x’L,y’L)

P = (x,y,z)

OR

i

j

Simplest Case

• Image planes of cameras are coplanar

• Focal lengths equal

• Optical centres are at same height (i.e. x-axes collinear)

• Intersection of optical axes at infinity (i.e. z-axes parallel)

k

(x’R,y’R)

Computer Vision / Mid-Level Vision / Stereo and Depth 6

baseline B

Image formation reminder Virtual image

P’

k

f

P = (x,y,z)

O

i

j

3D scene point P is projected to a point P’ on the image, such that:

P’=(x’,y’)=(fx, fy) zz

Assuming that the image centre is (0,0) [see lecture 2]

Computer Vision / Mid-Level Vision / Stereo and Depth

7

(x’,y’) (0,0)

Stereo: coplanar cameras

OL

i

j

k

(x’L,y’L)

P = (x,y,z) w.r.t. OL

OR

i

j

Image formation for two cameras:

P projects to (x’R, y’R) and (x’L, y’L)

Note: Because x-axes of cameras are collinear, y’L = y’R

k

(x’R,y’R)

Computer Vision / Mid-Level Vision / Stereo and Depth 8

baseline B

Stereo: coplanar cameras

OL

i

j

(x’R,y’R)

(x’L,y’L)

P = (x,y,z) w.r.t. OL

k

OR

i

j

k

(x’L,y’L)=(fx, fy) zz

(x’R,y’R)=(f(x−B), fy) zz

Using the coordinate system of the left camera (since xR=xL-B)

Computer Vision / Mid-Level Vision / Stereo and Depth 9

baseline B

Stereo: coplanar cameras

OL

i

j

(x’R,y’R)

(x’L,y’L)

P = (x,y,z) w.r.t. OL

k

OR

i

j

k

(x’L,y’L)=(fx, fy) zz

(x’R,y’R)=(f(x−B), fy) zz

Disparity, d=x’L−x’R= fx−f(x−B)= fB z=f Bd zzz

Computer Vision / Mid-Level Vision / Stereo and Depth

10

baseline B

Disparity

Depth is inversely proportional to disparity.

B d

then we can calculate the depth of a point.

Even without the baseline, we can know the relative depths of points from their relative disparities.

z=f

If the baseline distance is known, and we can measure the disparity,

Computer Vision / Mid-Level Vision / Stereo and Depth 11

Disparity

The difference vector of the image coordinates of two corresponding points.

NOTE

CORRESPONDING POINTS

DISPARITY VECTOR

● ● ● ●

the disparity, d, of a point is a 2D vector.

disparity is measured in pixels and can be positive or negative

a pair of stereo images defines a field of disparity vectors (a disparity map) For coplanar cameras disparity is horizontal only

Computer Vision / Mid-Level Vision / Stereo and Depth 12

SUPERIMPOSED IMAGES

Disparity / depth map: example

left image right image

depth map

light = close, dark = far

Computer Vision / Mid-Level Vision / Stereo and Depth

13

Disparity / depth map: example

left image

right image

False shallow region caused by false matches

Computer Vision / Mid-Level Vision / Stereo and Depth

14

depth map

light = close, dark = far

The stereo correspondence problem

To measure disparity, it is necessary to find corresponding points in the stereo pair of images.

To solve the stereo correspondence problem, we can use:

• Correlation-based methods

yield dense disparity maps: a disparity value at each pixel.

• Feature-based methods

yield sparse disparity maps: a disparity value at interest points

only.

Computer Vision / Mid-Level Vision / Stereo and Depth 15

The stereo correspondence problem

To measure disparity, it is necessary to find corresponding points in the stereo pair of images.

Basic requirements to be able to solve the correspondence problem: 1. Most scene points visible in both images

2. Corresponding image regions appear “similar”

These assumptions hold if:

• The distance of the 3D point from the cameras is much larger than the baseline: z >> B

Computer Vision / Mid-Level Vision / Stereo and Depth 16

The stereo correspondence problem

To measure disparity, it is necessary to find corresponding points in the stereo pair of images.

As we saw in the previous lecture, solving the correspondence problem is not easy.

However, we can use knowledge about the stereo camera system to help find a solution…

Computer Vision / Mid-Level Vision / Stereo and Depth 17

Stereo Constraints on Correspondence

Epipolar constraint

For coplanar cameras, y’L = y’R so 2D search can be reduced to a 1D search along the “epipolar” line (= the corresponding row of pixels for

coplanar cameras).

Computer Vision / Mid-Level Vision / Stereo and Depth 18

Stereo Constraints on Correspondence

Maximum disparity constraint

Length of search region depends on the maximum expected disparity, often predictable geometrically (dmax = fB/zmin).

x’ x’

For each point (x’, y’) in the left image, search for its corresponding point between (x’-dmax,y’) and (x’+dmax,y’) in the right image.

Computer Vision / Mid-Level Vision / Stereo and Depth 19

Stereo Constraints on Correspondence

Continuity

Neighbouring points should have similar disparities, because the environment is made of continuous surfaces over which depth varies smoothly.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

20

right image left image

Stereo Constraints on Correspondence

Continuity

The exception is at discontinuities where depth (and hence disparity) can change suddenly.

object Centre of left camera

object

Centre of right camera

right image left image

Computer Vision / Mid-Level Vision / Stereo and Depth

21

Stereo Constraints on Correspondence

Uniqueness

A location in one image should only match a single location in the other image.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

22

right image left image

Stereo Constraints on Correspondence

Uniqueness

The exception is when a surface lies along a line-of-sight for one camera (in this case one location may match many locations).

If fact any inclined surface may project to n pixels in one image and m pixels in the other image (with m ≠ n).

Centre of left camera

object

Centre of right camera

right image left image

Computer Vision / Mid-Level Vision / Stereo and Depth

23

Stereo Constraints on Correspondence

Ordering

Matching points along corresponding epipolar lines should be in the same order.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

24

right image left image

Stereo Constraints on Correspondence

Ordering

The exception is when objects have different depths.

Centre of left camera

object

Centre of right camera

right image left image

Computer Vision / Mid-Level Vision / Stereo and Depth

25

Correspondence problems

Correspondence is fundamentally ambiguous, i.e. there are many possible solutions.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

26

right image left image

Correspondence problems

Correspondence is fundamentally ambiguous, i.e. there are many possible solutions.

Centre of left camera

object

Centre of right camera

right image left image

Computer Vision / Mid-Level Vision / Stereo and Depth

27

Correspondence problems

Correspondence is fundamentally ambiguous, i.e. there are many possible solutions.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

28

right image left image

Correspondence problems

Correspondence is fundamentally ambiguous, i.e. there are many possible solutions.

We are trying to use imperfect constraints to narrow down these many potential solutions to the correct one.

Centre of left camera

object

Centre of right camera

Computer Vision / Mid-Level Vision / Stereo and Depth

29

right image left image

Correspondence problems

Some points in each image will have no corresponding points in the other image:

1. due to occlusion (e.g. )

2. the cameras might have different fields of view (e.g. )

Centre of left camera

object

Centre of right camera

right image left image

Computer Vision / Mid-Level Vision / Stereo and Depth

30