Conformance and compliance testing

Conformance testing is an element of conformity assessment, and also known as compliance testing, or type testing - is testing or other activities that determine whether a process, product, or service complies with the requirements of a specification, technical standard, contract, or regulation.

Web standards

WebAssembly

WebAssembly testsuite

QUIC: A UDP-Based Multiplexed and Secure Transport

quic-tracker

HTML (HyperText Markup Language)

WebSocket

autobahn

WebCL

https://github.com/KhronosGroup/WebCL-conformance

WebRTC

https://webrtc.org/testing/conformance/

ECMAScript

ECMAScript Language Specification - https://tc39.es/ecma262/ test262

RFC 2616: Hypertext Transfer Protocol – HTTP/1.1

HTTPLint

RFC 7540: Hypertext Transfer Protocol Version 2 (HTTP/2)

There are two test suites: http2-test and h2spec.

RFC 4791: Calendaring Extensions to WebDAV (CalDAV)

https://github.com/apple/ccs-caldavtester

RFC 4918: HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)

http://www.webdav.org/neon/litmus/

RFC 1738: Uniform Resource Locators (URL)

Hypothesis + lark-grammars

RFC 2396: Uniform Resource Identifiers (URI): Generic Syntax

Hypothesis + lark-grammars

RFC 2397: The “data” URL scheme

Hypothesis + lark-grammars

RFC 5545: Internet Calendaring and Scheduling Core Object Specification (iCalendar)

Hypothesis + lark-grammars

Robots Exclusion Protocol

A file robots.txt used by websites primarily to manage crawler traffic to site. For a long time format has differences in descriptions provided by Google and Yandex in their documentations for webmasters. For example Yandex describes an option “Crawl-delay” but there is no such option in documentation provided by Google. In 2019 Google had an attempt to formalize the Robots Exclusion Protocol Specification and they published a draft, that was expired in Jan 2, 2020. However proposed specification has a grammar that can be used for testing parsers and producers on robots.txt files. For such purposes once can use Hypothesis and lark-grammars or robotstxt-spec-test.

Structured Query Language (SQL)

POSIX

Video

Network

TLS

Samba

Kerberos

Windows Protocol TestSuites

RDP (Remote Desktop Protocol)

Windows Protocol TestSuites

memcached

memcapable

A Stateless Transport Tunneling Protocol for Network Virtualization (STT)

stt

RFC 4960: Stream Control Transmission Protocol

sctp-tests

Signaling System 7 (SS7)

  • RFC 4666: Signaling System 7 (SS7) Message Transfer Part 3 (MTP3) - User Adaptation Layer (M3UA)

  • RFC 3331: Signaling System 7 (SS7) Message Transfer Part 2 (MTP2) - User Adaptation Layer

  • RFC 3686: Signalling Connection Control Part User Adaptation Layer (SUA)

  • RFC 4165: Signaling System 7 (SS7) Message Transfer Part 2 (MTP2) - User Peer-to-Peer Adaptation Layer (M2PA)

  • m2pa-testtool

  • sua-testtool

  • m2ua-testtool

  • m3ua-testtool

RFC 3261: SIP: Session Initiation Protocol

TTCN-3 test suite |

RFC 1035: Domain Names - Implementation And Specification

DNS Compliance Tests

IPSec

TCP (Transmission Control Protocol)

UDP

packetdrill testcases

SMTP

SMTP smtpscript

MQTT

Eclipse IoT-Testware

Data Formats

IEEE 754

There is a test suite for IEEE 754R compliance published by IBM. It contains test vectors arranged in files. Each file represents the test vectors for a model described in the document. Documentation, IBM Fpgen Floating-Point Test Generator.

There is a FPSpy, a tool that provide mechanisms for analyzing floating point arithmetic as it executes, making it possible to find and track problematic operations in unmodified x64 application binaries on Linux, including those using thread- and process-level parallelism. FPSpy can observe application behavior without any cooperation from the application or developer, and can potentially be deployed as part of a job launch process. Tool published as a part of research described in “Spying on the Floating Point Behavior of Existing, Unmodified Scientific Applications”. fpspy

Few C programs to test the floating-point arithmetic of your machine: Floating-Point Arithmetic Test Programs

Zoned Device ATA Command Set (ZAC)

https://zonedstorage.io/tests/zbc-tests/

RFC 822: Standard for the Format of ARPA Internet Text Messages

Hypothesis + lark-grammars

TAP (Test Anything Protocol)

TAP 13 TAP 14 Hypothesis + lark-grammars

UTF-8

UTF-8 decoder capability and stress test

YAML

YAML, Hypothesis + lark-grammars

RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format

https://json.org JSON Schema https://github.com/nst/JSONTestSuite JSON-Schema-Test-Suite

https://github.com/json-schema-org/JSON-Schema-Test-Suite

TOML TOML toml-test

S3

S3 compatibility tests

OPDS

OPDS (Open Publication Distribution System) OPDS Test Catalog

Markdown

Markdown is a simple widely-used text markup format. It was intended to be as easy-to-read and easy-to-write as is feasible. Initially it had a limited syntax, that was later was extended by additional syntax (like footnotes, math support etc). There are two testsuites that allows to test parsers and generators of texts with Markdown markup: mdtest and MarkdownTest.

GEDCOM

Hardware

ACPI

RISC

There is a RISC-V compliance testsuite - https://github.com/riscv/riscv-compliance.

There is a MicroTesk. MicroTESK is a reconfigurable and extendable framework for generating test programs for functional verification of microprocessors. MicroTESK allows automatically constructing test program generators based on formal specifications of microprocessor architectures. MicroTESK supports a wide range of architectures including RISC, CISC, VLIW, and DSP. MicroTESK supports online test program generation.

USB (Universal Serial Bus)

UEFI

Теги: softwaretestingfeeden

Новые

Все посты →