I needed something called camera calibration, so I tried it. Note: I am self-taught and may contain inaccurate statements.
There is an expression that can convert the position in space to the position on the image, but this requires a camera parameter. There are different types of camera parameters, which can be broadly divided into internal parameters and external parameters. There is also a lens distortion coefficient (distortion), which I don't know if it is included in the internal parameters.
Internal parameters are "parameters determined by the camera". ・ Focal length f ・ Physical spacing of pixels δ u </ sub>, δ v </ sub> -Image center in image coordinates relative to normalized coordinates c u </ sub>, c v </ sub>
External parameters are "parameters of camera position and orientation with respect to the world coordinate system". ・ Rotation matrix R 3 </ sub> ・ Translation vector t 3 </ sub>
Now, camera calibration is to find the parameters of this camera. The internal and external parameters obtained at this time are collectively called the camera matrix or perspective projection matrix. Determine the parameters by applying the spatial points whose positions are known in advance and the projection points (that is, the coordinates on the image) on the image to the matrix formula.
Specifically, take a picture while moving the calibration target (such as a chess board). Since the length of the chessboard grid is known, the distance between the grid points is known and its position in space is determined. (Conversely, it determines the world coordinate system) It also gets the coordinates of the grid points on the image. (Acquisition of image coordinates) Take multiple images and use the least squares method to find the parameters.
Quote (reference) Source: "Digital image processing [revised new edition]", the expression has been modified for convenience. Reference: Camera calibration and 3D reconstruction --OpenCV.jp
This time, I calibrated the camera with the chest board, which I think is relatively easy, according to the following rough procedure.
For the code, see OpenCV_CameraCalibration --GitHub please.
I will explain each in a little more detail.
Arbitrarily set the world coordinate system so that the plane of the flat calibration target (chest board) is Z = 0. At this time, measure the spacing between the grid points and set the points in mm. I'm calling here.
List<Mat> objectPoints = getObjectPoints(outputFindChessboardCorners.size(), patternSize); //Three-dimensional coordinates of chessboard corners(z=0), For the number of captured images.
OpenCV_CameraCalibration_L68 - GitHub
The contents of getObjectPoints () are as follows.
public List<Mat> getObjectPoints(int size, Size patternSize) {
List<Mat> objectPoints = new ArrayList<>();
for (int i = 0; i < size; i++) {
objectPoints.add(getObjectPoint(patternSize));
}
return objectPoints;
}
public MatOfPoint3f getObjectPoint(Size patternSize) {
MatOfPoint3f objectPoint = new MatOfPoint3f();
List<Point3> objectPoint_ = new ArrayList<>();
// final Size patternSize = new Size(6, 9); //Number of corners to explore
for (int row = 0; row < patternSize.height; row++) {
for (int col = 0; col < patternSize.width; col++) {
objectPoint_.add(getPoint(row, col));
}
}
objectPoint.fromList(objectPoint_);
return objectPoint;
}
public Point3 getPoint(int row, int col) {
final double REAL_HEIGHT = 20.0, REAL_WIDTH = 20.0;
return new Point3(col * REAL_WIDTH, row * REAL_HEIGHT, 0.0); //Maybe x, y,z looks like this.
}
OpenCV_CameraCalibration_L96 - GitHub
I wrote a method called findChessboardCorners () myself. The call looks like this:
List<Mat> outputFindChessboardCorners = new ArrayList<>();
try (DirectoryStream<Path> ds = Files.newDirectoryStream(picFolderPath)) {
for (Path path : ds) {
System.out.println(path.toString());
final Optional<Mat> outputMat = findChessboardCorners(path.toString(), imagePoints, patternSize);
if (outputMat.isPresent()) {
outputFindChessboardCorners.add(outputMat.get());
System.out.println("successful to find corners.");
} else {
System.err.println("unsuccessful to find corners.");
}
}
} catch (IOException ex) {
ex.printStackTrace();
}
OpenCV_CameraCalibration_L50 - GitHub
The contents of findChessboardCorners () are as follows. I used a method called Calib3d.findChessboardCorners (). In addition, the returned Mat contains an image that Calib3d.findChessboardCorners () was able to execute normally.
public Optional<Mat> findChessboardCorners(String picPathString, List<Mat> imagePoints, Size patternSize) {
Mat inputMat = Imgcodecs.imread(picPathString);
Mat mat = inputMat.clone();
// final Size patternSize = new Size(6, 9); //Number of corners to explore
MatOfPoint2f corners = new MatOfPoint2f(); // in,Receives a vector of the two-dimensional coordinates of the detected corner.
Imgproc.cvtColor(inputMat, inputMat, Imgproc.COLOR_BGR2GRAY);
final boolean canFindChessboard = Calib3d.findChessboardCorners(inputMat, patternSize, corners);
if (!canFindChessboard) {
System.err.println("Cannot find Chessboard Corners.");
return Optional.empty();
}
imagePoints.add(corners);
Calib3d.drawChessboardCorners(mat, patternSize, corners, true);
Path picPath = Paths.get(picPathString);
Path folderPath = Paths.get("S:\\CameraCaliblation\\2018-12-31_output");
Path path = Paths.get(folderPath.toString(), picPath.getFileName().toString());
if (!Files.exists(folderPath) || !Files.isDirectory(folderPath)) {
try {
System.out.println("There was no folder, so it is createing a folder. : " + folderPath.toString());
Files.createDirectory(folderPath);
} catch (IOException ex) {
ex.printStackTrace();
}
}
Imgcodecs.imwrite(path.toString(), mat);
return Optional.of(inputMat);
}
Calib3d.calibrateCamera () with the coordinates of the grid points in the space prepared earlier and the coordinates of the grid points of the chest board on the image obtained earlier as arguments. /camera_calibration_and_3d_reconstruction.html#cv-calibratecamera). Also, since this method returns the values of the camera matrix and lens distortion coefficient, create a variable to store them.
//What to receive
Mat cameraMatrix = new Mat(), distortionCoefficients = new Mat();
List<Mat> rotationMatrixs = new ArrayList<>(), translationVectors = new ArrayList<>();
Calib3d.calibrateCamera(objectPoints, imagePoints, imageSize,
cameraMatrix, distortionCoefficients, rotationMatrixs, translationVectors);
OpenCV_CameraCalibration_L81 - GitHub
I was able to get the camera matrix and lens distortion factor, but it's tedious and wasteful to do this every time, so let's save the matrix values. In the case of C and Python, there is something that inputs and outputs the Mat value on the OpenCV side, but since it is not the Java version, I made it myself this time. I used the XML input / output found in the Java standard library.
Map<String, Mat> exportMats = new HashMap<>();
exportMats.put("CameraMatrix", cameraMatrix);
exportMats.put("DistortionCoefficients", distortionCoefficients);
final Path exportFilePath = Paths.get("S:\\CameraCaliblation\\CameraCalibration_2018-12-31.xml");
MatIO.exportMat(exportMats, exportFilePath);
OpenCV_CameraCalibration_L87 - GitHub
The contents of MatIO are OpenCV_CameraCalibration_MatIO.java --GitHub Please refer to.
For the time being, using this, [Posture estimation of AR marker of ArUco](https://qiita.com/smk7758/items/1ca1370f78cad1233d02#%E3%82%AB%E3%83%A1%E3%83%A9% E3% 82% 92% E7% 94% A8% E3% 81% 84% E3% 81% 9F% E3% 83% AA% E3% 82% A2% E3% 83% AB% E3% 82% BF% E3% 82% A4% E3% 83% A0marker% E5% A7% BF% E5% 8B% A2% E6% 8E% A8% E5% AE% 9A) and it worked fine. I am glad that the juniors of our information group also understood and implemented it and succeeded.
・ OpenCV --How to calibrate the camera --Pynote ・ Camera Calibration (OpenCV 1.0) --OpenCV.jp -Image processing and recognition by OpenCV and Visual C ++ -Code for camera calibration (obtaining internal parameters and distortion coefficient of one camera) with python and OpenCV (Plagiarism) ・ Camera calibration -Camera calibration with OpenCV 2.4.11 & C ++-Technical singularity -Camera Calibration and 3D Reconstruction --OpenCV Official-There is a good explanation if you look at the official website again. -[CC_Controller.java --opencv-java / camera-calibration (GitHub)](https://github.com/opencv/opencv/blob/master/samples/android/camera-calibration/src/org/opencv/samples/ cameracalibration / CameraCalibrator.java) --It was helpful for the Java version. ・ Cameracalibration --opencv (GitHub)
Recommended Posts