Automating mobile gestures

• • • • • • Note for W3C actions
        • An Overview of the TouchAction / MultiAction API
        • • TouchAction
        • MultiTouch
        • Bugs and Workarounds
        • • Scrolling

While the Selenium WebDriver spec has support for certain kinds of mobile interaction, its parameters are not always easily mappable to the functionality that the underlying device automation (like UIAutomation in the case of iOS) provides. To that end, Appium implements the new TouchAction / MultiAction API defined in the newest version of the spec (https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#multiactions-1). Note that this is different from the earlier version of the TouchAction API in the original JSON Wire Protocol.

尽管'Selenium WebDriver'规定支持某些类型的移动交互，
它的参数并不是总能轻易的映射到underlying device automation
(比如iOS中的UIAutomation).
为此，Appium实现了规范最新版本中定义新的TouchAction / MultiAction API.
（https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html#multiactions-1）
注意，这与最初的JSON Wire Protocol的TouchAction API不同

These APIs allow you to build up arbitrary gestures with multiple actuators. Please see the Appium client docs for your language in order to find examples of using this API.

这些APIs允许你使用多个执行器建立任意收手势.
请参阅Appium client文档了解你的语言以便你使用API查找示例

Note for W3C actions

W3C actions is also available in some drivers such as XCUITest, UIA2, Espresso and Windows. W3C actions are implemented to the best of the limitations of the operating systems’ test frameworks. e.g. WDA cannot handle zero wait PR.

W3C action也可以在一些驱动中中使用，
例如XCUITest， UIA2,Espresso and Windows.
W3C action实现时最大限度利用了操作系统测试框架.
例如WDA无法处理的零等待PR.

API doc and API docs of each client help to understand how to call them.

An Overview of the TouchAction / MultiAction API

TouchAction

TouchAction objects contain a chain of events.

TouchAction objects包含了一系列事件

In all the appium client libraries, touch objects are created and are given a chain of events.

在所有的Appium client libraries,
touch objects是创建并被赋予一连串事件的

The available events from the spec are: * press * release * moveTo * tap * wait * longPress * cancel * perform

规定中可用的事件有：
 * press * release * moveTo * tap * wait * longPress * cancel * perform

Here’s an example of creating an action in pseudocode:

下面是一个建立在伪代码中的示例：

TouchAction().press(el0).moveTo(el1).release()

The above simulates a user pressing down on an element, sliding their finger to another position, and removing their finger from the screen.

以上模拟器了用户按下一个元素，
把手机滑动到另一个位置，
并从屏幕上移开他的手指.

Appium performs the events in sequence. You can add a wait event to control the timing of the gesture.

Appium performs按顺序执行事件.
你可以添加等待事件来控制手势的计时

moveTo coordinates are now absolute to the current position. For example, dragging from 100,100 to 200,200 can be achieved by:

moveTo坐标与当前位置是绝对关系.
例如，
拉一个100,100 to 200,200通过以下方式实现：

.press(100,100) // Start at 100,100
.moveTo(200,200) // Passing absolute values of 200,200 ending up at 200,200

The appium client libraries have different ways of implementing this, for example: you can pass in coordinates or an element to a moveTo event. Passing both coordinates and an element will treat the coordinates as relative to the element’s position, rather than absolute.

Appium client libraries有不同的实现方法.
例如：你可以将坐标或元素传递给moveTo事件.
传递两个坐标和元素将被被视为相对元素的位置.

Calling the perform event sends the entire sequence of events to appium, and the touch gesture is run on your device.

调用perform事件将整个事件序列发送给Appium，
并且触摸手势将在你的设备上执行.

MultiTouch

MultiTouch objects are collections of TouchActions.

多点触控对象是触控动作的集合

MultiTouch gestures only have two methods, add, and perform.

多点触控手势仅有两种方法`add`和`perform`

add is used to add another TouchAction to this MultiTouch.

在这个多点触控中add将用于另一个TouchAction

When perform is called, all the TouchActions which were added to the MultiTouch are sent to appium and performed as if they happened at the same time. Appium first performs the first event of all TouchActions together, then the second, etc.

当调用'perform'时，
如果他们同事发生
所有的TouchActions将被添加到多点触控系统中的数据被发送到Appium并被执行
Appium首先执行第一个事件或所有的TouchActions事件，
然后执行第二个事件...etc

Pseudocode example of tapping with two fingers:

用两个手指轻敲的伪代码示例：

action0 = `TouchAction`().`tap`(el)
action1 = `TouchAction`().`tap`(el)
'MultiAction'().'add'(action0).'add'(action1).'perform'()

Bugs and Workarounds

An unfortunate bug exists in the iOS 7.0 - 8.x Simulators where ScrollViews, CollectionViews, and TableViews don’t recognize gestures initiated by UIAutomation (which Appium uses under the hood for iOS). To work around this, we have provided access to a different function, scroll, which in many cases allows you to do what you wanted to do with one of these views, namely, scroll it!

在 iOS 7.0 - 8.x Simulators存在一个不好的'bug'，
'ScrollViews', 'CollectionViews', and 'TableViews'都不能识别UIAutomation创建的手势
(Appium在iOS的底层所用的框架).
为了解决这个问题，
我们提供了一个不同的接入函数'scroll',
在大多数情况下允许你去做你想做的操作
scroll it

Scrolling

To allow access to this special feature, we override the execute or executeScript methods in the driver, and prefix the command with mobile:. See examples below:

要允许访问特殊功能,
我们重写驱动程序中的execute或executeScript在driver中，
并在命令前面加上'mobile'，
参加以下示例

To scroll, pass direction in which you intend to scroll as parameter.

为scroll,请传递要作为参数滚动的方向.

Java	JavascriptExecutor js = (JavascriptExecutor) driver; HashMap<String, String> scrollObject = new HashMap<String, String>(); scrollObject.put("direction", "down"); js.executeScript("mobile: scroll", scrollObject);
Python	driver.execute_script("mobile: scroll", {"direction": "down"})
Javascript	driver.execute('mobile: scroll', {direction: 'down'});
Ruby	execute_script 'mobile: scroll', direction: 'down'
C#	Dictionary<string, string> scrollObject = new Dictionary<string, string>(); scrollObject.Add("direction", "down"); ((IJavaScriptExecutor)driver).ExecuteScript("mobile: scroll", scrollObject));
PHP	p a r a m s = a r r a y ( a r r a y ( ‘ ′ d i r e c t i o n ′ ‘ = > ‘ ′ d o w n ′ ‘ ) ) ; < b r / > params = array(array(`'direction'` => `'down'`));<br/> params=array(array(‘′direction′‘=>‘′down′‘));<br/>driver->executeScript("mobile: scroll", $params);

Sample to scroll using direction and element.

使用方向和元素去滚动示例

Java	JavascriptExecutor js = (JavascriptExecutor) driver; HashMap<String, String> scrollObject = new HashMap<String, String>(); scrollObject.put("direction", "down"); scrollObject.put("element", ((RemoteWebElement) element).getId()); js.executeScript("mobile: scroll", scrollObject);
Python	driver.execute_script("mobile: scroll", {"direction": "down", element: element.getAttribute("id")})
Javascript	driver.execute('mobile: scroll', {direction: 'down', element: element.value.ELEMENT});
Ruby	execute_script 'mobile: scroll', direction: 'down', element: element.ref
C#	Dictionary<string, string> scrollObject = new Dictionary<string, string>();

scrollObject.Add(“direction”, “down”);
scrollObject.Add(“element”, <element_id>);
((IJavaScriptExecutor)driver).ExecuteScript(“mobile: scroll”, scrollObject));
PHP
JavascriptExecutor js = (JavascriptExecutor) driver;
HashMap<String, String> scrollObject = new HashMap<String, String>();
scrollObject.put(“direction”, “down”);
scrollObject.put(“element”, ((RemoteWebElement) element).getId());
js.executeScript(“mobile: scroll”, scrollObject);
Swiping

This is an XCUITest driver specific method that is similar to scrolling (for reference, see https://developer.apple.com/reference/xctest/xcuielement).

This method has the same API as Scrolling, just replace “mobile: scroll” with “mobile: swipe”

Automating Sliders

iOS

Java
// java
// slider values can be string representations of numbers between 0 and 1
// e.g., “0.1” is 10%, “1.0” is 100%
WebElement slider = driver.findElement(By.xpath("//window[1]/slider[1]"));
slider.sendKeys(“0.1”);
Android

The best way to interact with the slider on Android is with TouchActions.