LEVELS OF TESTING - unit (component, module) * white box * done by programmer or white box tester * black box: feature/function/component - integration * white/black/gray box testing - makes sense - system * testing the product with no regard to where it fail - acceptance/user acceptance (beta testing, crowd testing) * done by SME on behalf of user * the last testing done before shipping the product TESTING TYPES - Build Acceptance: * Build Acceptance: making sure the that build is good enough to go to QA * Making sure the build is in testable condition * How: the most basic functions are exercized * Same as: smoke testing, sanity check * The most frequently executed test (#2 - regression) * Best candidate for test automation * Version (released to the public) * Build (for internal record) * Release (to be released to the public) ** Build 29.9076 - release candidate * Build Acceptance: making sure the that build is good enough to go to QA - Black Box testing: * Testing done from user perspective (no access to source code used) * Not about WHO does it, but its is about HOW it is done * Programmer: a) coding; b) black box testing; c) white box testing * Black Box Tester: a) black box testing; b) white box testing; c) gray box testing * Bugs are identified and reported from user perspective * Could be manual or automated (still black box) GRAY BOX TESTING: * Extension of BLACK BOX testing * Using structural, design, and environment information (complete or incomplete) to expand or focus black box testing and to enhance testing productivity * by using appropriate methods and tools. * a lot of web testing is done in gray box area * AVOID: Combination of black and white box * Examples: REST API/Web Services, database records, log files, protocols WHITE BOX TESTING: * done at the source code level * executed mostly by either developers or white box testers * Unit test, API Testing Regression Testing * making sure nothing was broken as a result of modifying the code * неважно ручное-автоматизированное или black/gray/white box * ЛЮБОЕ изменение в коде (build), не сужаем определение до починки бага или написанию нового функционала * не смешиваем с проверкой ошибки на то, что она устранена (bug fix verification) * НА ПРАКТИКЕ: объем тестирования ограничен в связи с нехваткой времени и других ресурсов Performance testing * responsiveness (как быстро реагирует -> response time) * stability (performance degradation) * нагрузка измеряется в объеме данных В ЕДИНИЦУ ВРЕМЕНИ, количестве пользователей, количество операций * нагрузка фиксирована на заданном интервале времени * Пример: Email application (MAX = 1 mln per minute) - 700.000/5,000,000 per minute for 4 hours * Кто и как: тестеровщик, программист пишет утилиту, LoadRunner/JMeter - Load (700.000) - Stress testing (5,000,000) Volume Testing * has nothing to do with time - it is about volume Exploratory * Цель - лучше понять тестируемый продукт * Бессистемное использование * Планирование, исполнение, изучение-понимание * Ad Hoc ("Monkey Testing") - Планирование-исполнение-изучение ОДНОВРЕМЕННО End-to-End testing: * Подмножество System testing * Slang: Test Scenario - многоходовка = use case * Not limited to execution of use cases * Some times comes from Technical Support group Error handling testing (exception handling)? * Makes sense to developers, not black box testers * has nothing to do with error (warning, confirmation) messages given to users * ПРИМ: Error messages could be either CONFUSING or MISLEADING API = Application Programming Interface - Programming/Scripting language (Java, C++) - Third party libraries - Google API (libraries) - needs to be white box/API tested by Google people - Google Maps API (libraries) - needs to be API tested by Mapt team SDLC = conceptual model used in project management, which describes software development process in stages - analysis - requirements - design - coding/development (alpha, beta, gamma/golden/final release) - testing - documentation - maintenance Development Server (Environment) - там работают программисты QA Server (Environment) - тут работают тестировщики Production Server (Environment) - это для пользователей Release Acceptance Check List