Welcome to LZN's Blog!
Wind extinguishes a candle but energizes fire.-
Object Detection Using Tensorflow Models
Please scroll to the FINAL for Ultimate Solution!
Follow the procedure to install the libs and dependencies.
In the last step, testing the installation:
Running tests under Python 3.6.10: /users/b145872/project-dir/Anaconda3_2020/envs/tf1.14/bin/python [ RUN ] ModelBuilderTest.test_create_experimental_model [ OK ] ModelBuilderTest.test_create_experimental_model [ RUN ] ModelBuilderTest.test_create_faster_rcnn_model_from_config_with_example_miner [ OK ] ModelBuilderTest.test_create_faster_rcnn_model_from_config_with_example_miner [ RUN ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul [ OK ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul [ RUN ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul [ OK ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul [ RUN ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul [ OK ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul [ RUN ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul [ OK ] ModelBuilderTest.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul [ RUN ] ModelBuilderTest.test_create_rfcn_model_from_config [ OK ] ModelBuilderTest.test_create_rfcn_model_from_config [ RUN ] ModelBuilderTest.test_create_ssd_fpn_model_from_config [ OK ] ModelBuilderTest.test_create_ssd_fpn_model_from_config [ RUN ] ModelBuilderTest.test_create_ssd_models_from_config [ OK ] ModelBuilderTest.test_create_ssd_models_from_config [ RUN ] ModelBuilderTest.test_invalid_faster_rcnn_batchnorm_update [ OK ] ModelBuilderTest.test_invalid_faster_rcnn_batchnorm_update [ RUN ] ModelBuilderTest.test_invalid_first_stage_nms_iou_threshold [ OK ] ModelBuilderTest.test_invalid_first_stage_nms_iou_threshold [ RUN ] ModelBuilderTest.test_invalid_model_config_proto [ OK ] ModelBuilderTest.test_invalid_model_config_proto [ RUN ] ModelBuilderTest.test_invalid_second_stage_batch_size [ OK ] ModelBuilderTest.test_invalid_second_stage_batch_size [ RUN ] ModelBuilderTest.test_session [ SKIPPED ] ModelBuilderTest.test_session [ RUN ] ModelBuilderTest.test_unknown_faster_rcnn_feature_extractor [ OK ] ModelBuilderTest.test_unknown_faster_rcnn_feature_extractor [ RUN ] ModelBuilderTest.test_unknown_meta_architecture [ OK ] ModelBuilderTest.test_unknown_meta_architecture [ RUN ] ModelBuilderTest.test_unknown_ssd_feature_extractor [ OK ] ModelBuilderTest.test_unknown_ssd_feature_extractor ---------------------------------------------------------------------- Ran 17 tests in 0.427sNow we test the object detection script. As the model need to run on GPU cluster, we cannot simply use jupyter notebook. So convert to plain python code.
jupyter nbconvert --to python object_detection_tutorial.ipynbExcecute the python code, got:
ModuleNotFoundError: No module named 'object_detection'Compile protobufs and install the object_detection package:
cd models/research/ protoc object_detection/protos/*.proto --python_out=. pip install .Execute again, got:
tensorflow.python.framework.errors_impl.NotFoundError: models/research/object_detection/data/mscoco_label_map.pbtxt; No such file or directoryThis is the relative path problem caused by changing directory from ipython notebook.
Execute again, got:
File "object_detection_tutorial.py", line 98, in <module> detection_model = load_model(model_name) File "object_detection_tutorial.py", line 62, in load_model model = tf.saved_model.load(str(model_dir), None) File "/users/b145872/project-dir/Anaconda3_2020/envs/tf1.14/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func return func(*args, **kwargs) TypeError: load() missing 1 required positional argument: 'export_dir'Version Change:
model = tf.saved_model.load(export_dir=str(model_dir)) # change to: model = tf.saved_model.load_v2(export_dir=str(model_dir),tags=None)New error:
Traceback (most recent call last): File "object_detection_tutorial.py", line 191, in <module> show_inference(detection_model, image_path) File "object_detection_tutorial.py", line 172, in show_inference output_dict = run_inference_for_single_image(model, image_np) File "object_detection_tutorial.py", line 141, in run_inference_for_single_image num_detections = int(output_dict.pop('num_detections')) TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tensor'…….
Ultimate Solution
After multiple tests, we still cannot run the script on gpu smoothly, tf1.9 1.14 and 2.1 all failed. When I come back to the github page. I found the updated
ipynb… And this time, the tf2.1-based env can run it with GPU! Although there are still errors, we reinstalled the tf2.1 byconda --force-reinstall, everything goes nice!Updated 2020-05-10
-
Scratch Notes for Stanford CS107: Programming Paradigm Ch01-02
L1: Administrative Details
Notes
- C
- Assambly
- C++
- Concurrent Programming
- Scheme
- Python
C is more procedure-oriented, from main to sub function, and sub function to sub-sub function. “Procedure Paradiam”
Concurrent programming issue: The shared resource that two different processes to access to.
Scheme is in functional paradiam: rely on a return from a function to move forward! Thus you can program without side effects.
e.g. side effects in C/C++: Pass a data structure into a function by reference, and change it in the function, when return, it changed.
Python, with good web libraries.
Qustions
- Here talking about concurrent programming in one CPU? Seemingly in a Time-sharing system.
Glossary
handouts 讲义 mileage 里程 pertinent 相关的 synopsis 概要 assembly code 汇编代码 asterisks 星号 ampersands 连字号 miniature 微型L2: C/C++ Data Types - Interpretations
Notes
Low-level memory mechanisms.
C/C++ Data types |Data Type|Length| |—-|—-| |bool|1 byte| |char|1 byte| |short|2 byte| |int|4 byte*| |long|4 byte| |float|4 byte| |double|8 byte|
Technically, a Boolean can be mapped by a single bit in memory. Can engineer the compiler to do so.
Considering sign, the first bit in 1 — minus sign.
When 0000 0000 0000 0111 + 1000 0000 0000 0111, if the first bit is sign, the result is not right. To deal with this problem, use invert + 1 to represent negative values. Two’s complement(补码)
char ch = 'A'; short s = ch; cout << s << endl; OUTPUT: 65ch 1111 65 s 1111 65 int i = 2^21+2^10+2^1 short s = i OUTPUT: 1026short s = -1; int i = s;s 1111 1111 1111 1111 i 1111 1111 1111 1111 1111 1111 1111 1111 Floating things are wierd. See IEEE 754
Single (32 bits/ 4 bytes):
(-1)^sign(1.b22b21…b0)2^(e-127)
Double (64 bits/8 bytes):
Qustions
Precision varies with the magnitude of the represented number:
Single Precision: 10^N~10e(N-8) Double Precision: 10^N-10e(N-16)
int i =5; float f=i; cout<<f<<endl;5=>5.0=1.25*2^e # Totally different!
Be careful with the precision loss when convert
inttofloat!Doublemakes things better, merely better.Glossary
Updated 2020-04-26
-
The tee command combination in interactive runtime to redirect stdout/stderr stream
In an interactive application runtime, you may both hope the standard output (1) or error output (2) can shown both on the screen, and redirected to a file. The traditional way,
./xxx >& xxx.logdoes not work as you need to interact with the program. Here we need theteecomannd. Simply type:./xxx 2>&1 | tee xxx.logWill work for you. Here we decompose the command.
teeusing standard streams which reads standard input and writes it to both standard output and one or more files, effectively duplicating its input.2>&1redirect stderr to stdout (2>&1):|pipeline, a mechanism for inter-process communication using message passing. A pipeline is a set of processes chained together by their standard streams, so that the output text of each process (stdout) is passed directly as input (stdin) to the next one.tee xxx.logIn this case, the stdout from./xxxwill be redirected by the pipeline to tee, as stderr (2) has been already redirected to stdout (1), the tee command will take in both stdout and stderr.
That is how this command works!
Updated 2020-04-25