Google Test
test
Our great sponsors
Google Test | test | |
---|---|---|
67 | 9 | |
33,041 | 906 | |
2.6% | - | |
8.3 | 2.5 | |
8 days ago | 11 months ago | |
C++ | Python | |
BSD 3-clause "New" or "Revised" License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Google Test
-
Creating k-NN with C++ (from Scratch)
cmake_minimum_required(VERSION 3.5) project(knn_cpp CXX) include(FetchContent) FetchContent_Declare( googletest GIT_REPOSITORY https://github.com/google/googletest.git GIT_TAG release-1.11.0 ) FetchContent_MakeAvailable(googletest) FetchContent_Declare(matplotplusplus GIT_REPOSITORY https://github.com/alandefreitas/matplotplusplus GIT_TAG origin/master) FetchContent_GetProperties(matplotplusplus) if(NOT matplotplusplus_POPULATED) FetchContent_Populate(matplotplusplus) add_subdirectory(${matplotplusplus_SOURCE_DIR} ${matplotplusplus_BINARY_DIR} EXCLUDE_FROM_ALL) endif() function(knn_cpp_test TEST_NAME TEST_SOURCE) add_executable(${TEST_NAME} ${TEST_SOURCE}) target_link_libraries(${TEST_NAME} PUBLIC matplot) aux_source_directory(${CMAKE_CURRENT_SOURCE_DIR}/../lib LIB_SOURCES) target_link_libraries(${TEST_NAME} PRIVATE gtest gtest_main gmock gmock_main) target_include_directories(${TEST_NAME} PRIVATE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/../) target_sources(${TEST_NAME} PRIVATE ${LIB_SOURCES} ) include(GoogleTest) gtest_discover_tests(${TEST_NAME}) endfunction() knn_cpp_test(LinearAlgebraTest la_test.cc) knn_cpp_test(KnnTest knn_test.cc) knn_cpp_test(UtilsTest utils_test.cc)
-
Starting with C
Okay, time to start unit tests!!! We will use Unity Test Framework to do unit testing. It is one of widely used testing frameworks alongside with Check, Google Test etc. Just downloading source code, and putting it to the project folder is enough to make it work (that is also why it is portable).
-
Just in case: Debian Bookworm comes with a buggy GCC
Updating GCC (it happened to GoogleTest).
-
Automatically run tests, formatters & linters with CI!
Roy's project uses Google Test, a C++ testing framework. His testing setup is similar to mine as we both keep source files in one directory and tests in another. The key difference is that I can run the tests using the Visual Studios run button. It was fairly easy to write the new tests as there were existing ones that I could reference to check the syntax!
-
C++ Unit Testing Using Google Test - My Experience
The Google Test Documentation provides a primer for first-time users. The primer introduces some basic concepts and terminology, some of which I've been able to learn for this lab exercise.
-
Basic C++ Unit Testing with GTest, CMake, and Submodules
> git submodule add https://github.com/google/googletest.git > git submodule update --init --recursive
-
VS code + cmake + gtest?
cmake_minimum_required(VERSION 3.14) project(my_project) # GoogleTest requires at least C++14 set(CMAKE_CXX_STANDARD 14) set(CMAKE_CXX_STANDARD_REQUIRED ON) include(FetchContent) FetchContent_Declare( googletest URL https://github.com/google/googletest/archive/03597a01ee50ed33e9dfd640b249b4be3799d395.zip ) # For Windows: Prevent overriding the parent project's compiler/linker settings set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) FetchContent_MakeAvailable(googletest) enable_testing() add_executable( hello_test hello_test.cpp ) target_link_libraries( hello_test GTest::gtest_main ) include(GoogleTest) gtest_discover_tests(hello_test)
-
FetchContent with Multiple URLs
FetchContent\_Declare(googletestGIT\_REPOSITORY [[email protected]](mailto:[email protected]):googletest.git [https://github.com/google/googletest.git](https://github.com/google/googletest.git)GIT\_TAG release-1.12.1)FetchContent\_MakeAvailable(googletest)
-
CI/CD pipelines for embedded
Not sure about CppUnit but I can speak to my previous experience using the googletest framework which compiles your tests to an executable, and since it's a very simple framework we were able to cross-compile and run directly on our device. We just had to hook up a device to the server that was running the CI so it could flash it when needed. That basically meant that our process was:
- Basic CMake question regarding subdirectories
test
- Measuring Multitask Language Understanding
-
Mixtral 7B MoE beats LLaMA2 70B in MMLU
Sources [1] MMLU Benchmark (Multi-task Language Understanding) | Papers With Code https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu [2] MMLU Dataset | Papers With Code https://paperswithcode.com/dataset/mmlu [3] hendrycks/test: Measuring Massive Multitask Language Understanding | ICLR 2021 - GitHub https://github.com/hendrycks/test [4] lukaemon/mmlu · Datasets at Hugging Face https://huggingface.co/datasets/lukaemon/mmlu [5] [2009.03300] Measuring Massive Multitask Language Understanding - arXiv https://arxiv.org/abs/2009.03300
-
BREAKING: Google just released its ChatGPT Killer
With a score of 90.0%, Gemini Ultra is the first model to outperform human experts on MMLU (massive multitask language understanding), which uses a combination of 57 subjects such as math, physics, history, law, medicine and ethics for testing both world knowledge and problem-solving abilities.
-
[Colab Notebook] Launch quantized MPT-30B-Chat on Vast.ai using text-generation-inference, integrated with ConversationChain
One method for comparison is the MMLU https://arxiv.org/abs/2009.03300.
- Partial Solution To AI Hallucinations
- Announcing GPT-4.
-
Show HN: Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model
Because there are many benchmarks that measure different things.
You need to look at the benchmark that reflects your specific interest.
So in this case ("I wasn't impressed that 30B didn't seem to know who Captain Picard was") the closest relevant benchmark they performed is MMLU (Massive Multitask Language Understanding"[1].
In the LLAMA paper they publish a figure of 63.4% for the 5-shot average setting without fine tuning on the 65B model, and 68.9% after fine tuning. This is significantly better that the original GPT-3 (43.9% under the same conditions) but as they note:
> "[it is] still far from the state-of-the-art, that is 77.4 for GPT code-davinci-002 on MMLU (numbers taken from Iyer et al. (2022))"
InstructGPT[2] (which OpenAI points at as most relevant ChatGPT publication) doesn't report MMLU performance.
[1] https://github.com/hendrycks/test
[2] https://arxiv.org/abs/2203.02155
-
DeepMind's newest language model, Chinchilla (70B parameters), significantly outperforms Gopher (280B) and GPT-3 (175B) on a large range of downstream evaluation tasks
Benchmark result is 67.6% which is 7.6% improvement from Gopher. MMLU is multiple choice Q&A over various subjects. Questions can be found linked in this github repo (see data).
What are some alternatives?
Catch - A modern, C++-native, test framework for unit-tests, TDD and BDD - using C++14, C++17 and later (C++11 support is in v2.x branch, and C++03 on the Catch1.x branch)
mmfewshot - OpenMMLab FewShot Learning Toolbox and Benchmark
Boost.Test - The reference C++ unit testing framework (TDD, xUnit, C++03/11/14/17)
gpt-neo - An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
CppUTest - CppUTest unit testing and mocking framework for C/C++
RAD - RAD Expansion Unit for C64/C128
CppUnit - C++ port of JUnit
ut - C++20 μ(micro)/Unit Testing Framework
doctest - The fastest feature-rich C++11/14/17/20/23 single-header testing framework
elm-test-rs - Fast and portable executable to run your Elm tests
Unity Test API - Simple Unit Testing for C
benchmark - A microbenchmark support library