Morozov, Ivanov and Tikhonov regularization based LS-SVMs

The Ultimate Ownership of Western European CorporationsMara Faccio * and Larry H.P. Lang **(Forthcoming, Journal of Financial Economics)AbstractWe analyze the ultimate ownership and control of 5,232 corporations in 13 Western European countries. Fi r ms are typically widely held (36.93 percent) or family controlled (44.29 percent). Widely-held firms are more important in the U.K. and Ireland, family-controlled firms in continental Europe. Financial and large firms are more likely to be widely-held, while non-financial and small firms are more likely to be family-controlled. State control is important for larger firms in certain countries. Dual class shares and pyramids are used to enhance the control of the largest shareholders, but overall there are significant discrepancies between ownership and control in only a few countries.----------* Department of Finance and Business Economics, Mendoza College of Business, University of Notre Dame, Notre Dame, IN 46556-5646, U.S.A. Tel.: +1 219 631 5540; fax: +1 219 6315255; e-mail: mfaccio@.** Chinese University of Hong Kong, 225 LKK Building, Shatin, Hong Kong. Tel.: +852 2609 7761; e-mail: llang@.hk.We are grateful to Marco Bigelli, Lorenzo Caprio, Edith Ginglinger, Arun Khanna, John McConnell, Stefano Mengoli, Robert Pye, Gordon Roberts, Bill Schwert (the editor), George Tian, and especially Andrei Shleifer and an anonymous referee for providing helpful comments. We benefited from comments from workshop participants at the Nati onal University of Singapore, Washington University at St Louis, the Capacity Building Seminar in Manila (sponsored by the Asian Development Bank), the European Financial Management Association meeting (Athens), the European Finance Association meeting (London), the Financial Management Association meeting (Seattle), and the Scottish Institute for Research in Investment and Finance (Edinburgh). We also thank Bolsa de Valores de Lisboa, Commerzbank, Helsinki Media Blue Book, Hugin, the Union Bank of Switzerl and, and the Vienna Stock Exchange for generously providing us with their data sets. Mara Faccio acknowledges research support from Università Cattolica, Milan, and MURST (research grant #MM13572931). Larry Lang acknowledges research support from a Hong Kong Earmarked Grant.The Ultimate Ownership of Western European Corporations1. IntroductionRecent studies suggest that Berle and Means’ (1932) model of widely-dispersed corporate ownership is not common, even in developed countries.1 In fact, large shareholders control a significant number of firms in many countries, including developed ones. To examine ownership and control by large shareholders, La Porta, Lopez-de-Silanes and Shleifer (1999) traced the control chains of a sample of 30 firms in each of 27 countries. They documented the ultimate controlling owners and how they achieved control rights in excess of their ownership rights through deviations from the one-share-one-vote rule, pyramiding, and cross-holdings. Claessens, Djankov, and Lang (2000) carried out a similar task for 2,980 listed firms in 9 East Asian countries.2 They found significant discrepancies between ultimate ownership and control, allowing a small number of families to control firms representing a large percentage of stock market capitalization.This paper answers two questions. What is the structure of the ultimate ownership of Western European firms? What are the means by which owners gain control rights in excess of ownership rights? To answer these questions, we collect ultimate o wnership data for a sample of 5,232 listed firms in Austria, Belgium, Finland, France, Germany, Ireland, Italy, Norway, Portugal, Spain, Sweden, Switzerland, and the U.K. We include a large number of medium- and small-sized corporations, and we include both non-financial and financial companies. We measure ownership and control in terms of cash flow and voting rights. For example, if a family owns 25 percent of Firm X that owns 20 percent of Firm Y, then this family owns 5 percent of the cash flow rights of Firm Y – the product of the ownership stakes along the chain – and controls 20 percent of Firm Y – the weakest link along the control chain.Western European firms are most likely to be widely held (36.93 percent) or family controlled (44.29 percent). Widely-held firms are especially important in the U.K. and Ireland, while family control is more important in continental Europe. Widely-held firms are more important for financial and1 See Shleifer and Vishny (1997), Claessens et al. (2000) and Holderness et al. (1999).1large firms, while families are more important for non-financial and small firms. In certain countries, widely-held financial institutions also control a significant proportion of firms, especially financial firms. In some countries of continental Europe, the State also controls a significant proportion of firms, especially the largest. Widely-held corporations control few firms.We report the use of multiple classes of shares, pyramidal structures, holdings through multiple control chains and cross-holdings3, devices that give the controlling shareholders control rights in excess of their cash flow rights. Dual class shares are used by few firms in Belgium, Portugal, and Spain, but by 66.07 percent, 51.17 percent, and 41.35 percent of firms in Sweden, Switzerland, and Italy. Pyramids and holdings through multiple control ch ains are used to control only 19.13 and 5.52 percent of listed firms respectively, being less important for family-controlled firms and more important for firms controlled by the State and by widely-held financial institutions. 53.99 percent of European firms have only one controlling owner. More than two-thirds of the family-controlled firms have top managers from the controlling family. Overall, we find a substantial discrepancy between ownership and control in Belgium, Italy, Norway, Sweden, and Switzerland, but much less elsewhere.Our results for the 20 largest firms differ slightly from those of La Porta et al. (1999), in that we find fewer State-controlled firms and more widely held firms, fewer pyramids, and more dual class shares. Compared to the findings of Claessens et al. (2000) for East Asia, we find that: families control a higher proportion of firms; each family controls fewer firms on average; top families control a lower proportion of total stock market capitalization4; a higher proportion of family-controlled companies have family members in top management; and the largest shareholder is less often alone, but averages much higher cash flow rights, control rights and ratio of cash flow to voting rights. These differences may be due to weaker law enforcement in Asia that allows controlling owners to achieve effective control of a large number of firms by controlling and owning a smaller part of each firm.2 Hong Kong, Indonesia, Japan, South Korea, Malaysia, the Philippines, Singapore, Taiwan, and Thailand.3 Pyramiding occurs when the controlling shareholder owns one corporation through another which he does not totally own. Firm Y is held through “multiple control chains” if it has an ultimate owner who controls it via a multitude of control chains, each of which includes at least 5 percent of th e voting rights at each link. “Cross-holdings” means company Y directly or indirectly controls its own stocks.2Section 2 describes the data. Section 3 discusses ultimate ownership patterns. Section 4 discusses the means whereby owners gain control rights in excess of ownership rights and Section 5 measures the extent to which this has been achieved. Section 6 presents conclusions.2. Data2.1. Data sources and sample selectionSome of the p revious studies of corporate ownership and control, such as Lins and Servaes (1999a, 1999b), relied primarily on Worldscope. However, we found its coverage inadequate. For example, Worldscope included only 176 of 632 Spanish listed firms at the end of 1997.5 Moreover, ownership data was sometimes missing6. To ensure accuracy, we included only countries for which we could obtain alternative sources (especially primary or official) to permit cross-checking.7 This consideration excluded Luxembourg, Greece8, D enmark9 and Holland10 leaving 13 Western European countries that had comprehensive, reliable ownership data: Austria, Belgium, Finland, France, Germany, Ireland, Italy, Norway, Portugal, Spain, Sweden, Switzerland and the U.K. For these countries, Table 1 l i sts the data sources for ownership and multiple classes of shares. Data limitations confined our4 Japan stands apart from the rest of East Asia in this regard: the top Japanese family controls only 0.5 percent of total market capitalization5 In this case, we instead relied upon the Spanish Stock Exchange regulatory authority’s files (Comision Nacional del Mercado de Valores, 1998) which provides quarterly information on all shareholders with at least 5 percent of control rights, as well as directors’ ownership for all listed firms.6 In this case, Worldscope reports zero ownership stakes.7 We did not rely on Worldscope if we had an official data source (i.e., the Stock Exchange ownership files). When official data sources were not available, we collected data from alternative sources. We used Worldscope for ownership data only when information for a specific firm could not otherwise be identified.8 In addition, Greek shares are often held in bearer form, masking the identities of their owners.9 La Porta et al. (1999) used Hugin for Denmark. However, Hugin covers less than 20 percent of all listed firms and we could not locate information from other official sources10 The Stichting Toezicht Effectenverkeer (STE, the Securities Board of the Netherlands), responded to our request for data as follows: “The implementation of the 1992 Act has resulted in the list of disclosed notifications being no more than an historical overview that has been overtaken by countless events and no longer provides the desired transparency of the (Dutch) stock market. Furthermore, the file corruption will become greater with the passage of time. Given the above and the fact that the 19963sample to the period from 1996 to the end of 1999.11 This is not a significant restriction since ownership structures tend to be rather stable, as noted in La Porta et al. (1999).The European Corporate Governance Network (ECGN), too, has sponsored several studies on ownership structures within the European Union.12 However, compliance with the European Union directive on large shareholdings (88/627/EEC) restricts meaningful cross-country analysis with non-European Union countries. In particular, the ownership measures documented in those studies represent neither ultimate ownership nor ultimate control stakes. Specifically: they do not consider the use of multiple voting shares; they simply add up direct and indirect control stakes without tracing them to the ultimate owners; the controlling owner is defined as the one who controls an absolute majority (i.e., over 50 percent) of voting rights, or holds enough voting rights to have de facto control13,14 and the mechanisms used to secure control rights in excess of ownership rights are not systematically analyzed.[Insert Table 1 about here]Over the sample period, a total of 5,547 firms were listed in these 13 countries. Of these, we excluded 167 firms whose ownership was not recorded, 61 that use nominee accounts (mostly in the U.K. and Ireland, where firms are not required to disclose the identity of their “true” owners) and 87Act does not empower the STE to take any measures in this regard which would lead to an up-to-date overview, the STE does not issue this list to third parties.” See also de Jong et al. (1998).11 Data are from 1996 for France, Germany, Italy, Switzerland, and the U.K.; from 1997 for Spain and Portugal; from 1998 for Sweden a nd Norway; and from 1999 for Austria, Belgium, Finland, and Ireland.12 Becht and Boehmer (1998) for Germany; Bianchi, Bianco, and Enriques (1998) for Italy; Bloch and Kremp (1998) for France; Crespi-Cladera and Garcia-Cestona (1998) for Spain; de Jong, Kab ir, Marra, and Roell (1998) for the Netherlands; Renneboog (1998) for Belgium; Goergen and Renneboog (1998) for the U.K.13 For example, French regulation defines de facto control as occurring when a person or a legal entity owns, directly or indirectly, more than 40 percent of voting rights and no other partners or shareholders own a higher percentage directly or indirectly (Bloch and Kremp, 1998). This corresponds to a control threshold of 40 percent.14 To illustrate the bias that this definition introduces, consider the ultimate control structure of Montedison (Italy). Montedison has two shareholders with a stake above 2 percent: Compart with a stake of 33.45 percent and Mediobanca with a stake of 3.77 percent. Compart is indicated in the Italian supervisory authority’s files as the “ultimate” owner of Montedison. However, we found that Compart has three shareholders with stakes above 10 percent: Credit (11.01 percent), Cassa di Risparmio di Roma (10.14 percent), and Mediobanca (15.26 percent). According to our definition, Compart is the ultimate controlling shareholder of4affiliates of foreign firms (i.e., a foreign firm controls at least 50 percent of their voting rights) where we could not follow their ownership chain.15 This screening left 5,232 firms comprising 94.32 percent of the listed firms in the 13 countries.16 For these firms, our database records all owners who control at least 5 percent of voting rights. In France, Germany, and Spain, such owners must disclose their identity; the disclosure threshold is 2 percent in Italy and 3 percent in the U.K.The difficulty of organizing dispersed shareholders means that if the largest shareholder holds a substantial block of shares, then he has effective control. In line with earlier studies, we shall assume that 20 percent of the voting shares suffices to ensure control. We shall also discuss some cases that assume a control threshold of 10 percent. If no shareholder exceeds a given control threshold, then the firm is said to be “widely held” at that threshold. Table 2 sets out the screening process and lists the samples of firms analyzed in later tables.[Insert Table 2 about here]The exclusion of firms that use nominee accounts may overstate the proportion of widely-held firms in our sample. However, nominee accounts are the largest shareholders in only a small proportion of firms (less than 5 percent), so any bias is likely to be marginal. Ireland and the U.K. have the highest proportion of nominee accounts so this bias would be strongest there. These countries also have the highest proportions of widely-held firms, so the exclusion of firms using nominee accounts is unlikely to distort our cross-country comparisons.A shareholder of a corporation is said to be an ultimate owner at a given threshold if he controls it via a control chain whose links all exceed that threshold. If a firm has two owners with 12 percent of control rights each, then we say that the firm is half controlled by each owner at the 10 percent threshold, but that the firm is widely-held at the 20 percent threshold. In the case of a firm with two owners – a family with 20 percent of control rights and a widely-held corporation with 19 percentMontedison at the 20 percent threshold. However, at the 10 percent threshold, Mediobanca would be the largest ultimate owner of Montedison, with a 15.26 percent + 3.77 percent = 19.03 percent control stake.15 We retain the affiliates of foreign firms in our database whenever the holding firm is included in our sample.16 Our sample coverage of countries is: Austria: 100.00 (percent), Belgium: 89.04, Finland: 87.76, France: 89.26, Germany: 99.15, Ireland: 82.14, Italy: 100.00, Norway: 72.77, Portugal: 100.00, Spain: 100.00, Sweden: 94.96, Switzerland: 100.00, and the U.K.: 95.69. The low figure for Norway is due more to limited data coverage than to our screening.5of control rights – we would say that this firm is half controlled by each owner at the 10 percent threshold, but family-controlled at the 20 percent threshold.2.2. Calculation of cash flow rights and control rightsCorporate ownership is measured by cash flow rights, and control by voting rights. Ownership and control rights can differ because corporations can issue different classes of shares that provide different voting rights for given cash flow rights. Ownership and control rights can also differ because of pyramiding and holdings through multiple control chains.Firm Y is said to be controlled through “pyramiding” if it has an ultimate owner, who controls Y indirectly through an other corporation that it does not wholly control. For example, if a family owns 15 percent of Firm X, that owns 20 percent of Firm Y, then Y is controlled through a pyramid at the 10 percent threshold. However, at the 20 percent threshold, we would say t hat Firm Y is directly controlled by Firm X (which is widely-held at the 20 percent threshold) and no pyramiding would be recorded. If Firm X holds 100 percent of Firm Y, then again there is no pyramid. Pyramiding implies a discrepancy between the ultimate owner’s ownership and control rights. In the above example, the family owns 3 percent of the cash flow rights of Firm Y – the product of its ownership stakes along the control chain — but its control rights are measured by the weakest link in its control chain, i.e., 15 percent.Firm Y is said to be controlled through a “multiple control chain” if it has an ultimate owner who controls it via a multitude of control chains, each of which includes at least 5 percent of the voting rights at each link. In the previous example, suppose that the family also owns 7 percent of Firm Y directly. Then the family owns 10 percent of the cash flow rights of Firm Y (0.15 * 0.20 + 0.07) and controls 22 percent of its voting rights (min (0.15, 0.20) + 0.07).171817 A firm can be controlled by holdings through multiple control chains, even though it is not controlled by pyramiding. For example, suppose that Firm A controls 10 percent of B and 100 percent of C, which controls 15 percent of B. Since C is fully controlled by A in the control chain A-C-B, there is no pyramiding. However, Firm A controls Firm B directly and indirectly through Firm C, with control rights of 25 percent. We conclude that Firm A controls Firm B through multiple control chains bec ause: (1) Firm B has a controlling owner at the 20 percent level; (2) B is controlled via multiple control chains; and (3) all links in each chain involve at least 5 percent of the control rights.18 Claessens et al. (2000) defined “holdings through multiple control chain” as “cross-holdings”.6Firm Y is said to be controlled by a “cross-holding” at the 20 percent threshold if Firm X holds a stake in Firm Y of at least 20 percent, and Y holds a stake in Firm X of at least 20 percent, or if firm Y holds directly at least 20 percent of its own stocks.2.3. Types of ownershipThis section discusses our classification of ultimate owners into the following six types: Family: A family (including an individual), or a firm that is unlisted on any stock exchange.Widely-held financial institution: A financial firm (SIC 6000-6999) that is widely-held at the control threshold.State: A national government (domestic or foreign), local authority (county, municipality, etc.), or government agency.Widely-held corporation: A non-financial firm, widely-held at the control threshold.Cross-holdings: The firm Y is controlled by another firm, that is controlled by Y, or directly controls at least 20 percent of its own stocks.Miscellaneous: Charities, voting trusts, employees, cooperatives, or minority foreign investors.Where the ultimate owner of a corporation is an unlisted firm, we tried to trace its owners19 using all available data sources. We had incomplete success because most of our sample countries do not require unlisted firms to disclose their owners.20 If we failed to identify the owners of an unlisted19 In the case of the Fiat group, for example, we traced the ownership of La Rinascente back to the Agnelli family from Il Taccuino dell’Azionista, although we find two unlisted firms in its chain of control (namely, Carfin and Eufin). In fact, La Rinascente is controlled by Eufin (with a 32.8 percent ownership and a 40.51 percent control stake), which is wholly controlled by Ifil. Wer gehört zu wem is particularly useful in a number of cases in Germany. However, it helps us identify the owners for only 20 percent of unlisted German firms. For example, we find that TCHIBO Holding AG is the largest owner of Beiersdorf AG (a listed firm) with a 25.87 percent O&C stake. TCHIBO Holding, however, is an unlisted firm. We identified its ultimate owner: the Herz family, which has a 100 percent O&C stake. Another case is Heidelberger Zement AG, whose largest owner (with a 19.07 percent O stake and a 21.8 percent C stake) is Schwenk Gmbh, again an unlisted firm. We find that Schwenk is 90 percent owned and controlled by the Schwenk family, and 10 percent owned by the Babette family. In some cases, the ownership structure of unlisted firms is more complex. For example, the direct owner of Thyssen AG is Thyssen Beteiligunsverw Gmbh, which is unlisted. We find that this firm is 49.999 percent owned and controlled by Commerzbank (a listed, widely-held financial firm), and 49.981 percent owned and controlled by Allianz. A number of complex cases of ownership of unlisted firms were also reported by La Porta et al. (1999).20 One exception is the U.K., where the 3 percent disclosure rule also applies to unlisted firms. However, we were still not able to find ownership data for all unlisted firms.7firm, then we classified them as a family.21 Below, we offer both a general justification for this convention and statistical support for it in the largest European economies.Our database records the unlisted subsidiaries of widely-held corporation or financial institution, so any listed firm controlled by an unlisted firm that is controlled by a widely-held corporation or financial institution would be recorded as being controlled by the latter in our database. Thus, an unlisted firm that we identified as the ultimate controller of a listed firm is unlikely to be, in fact, controlled by a widely-held corporation or financial institution. Nor is the unlisted firm likely to be controlled by the State; insofar as the State controls firms, they tend to be listed. In any case, State ownership has decreased dramatically in Europe after the privatization wave of the 1990s. The low likelihood that an unlisted firm is, in fact, controlled by a widely-held corporation, widely-held financial institution or the State leaves families as the most likely controller of an unlisted firm. This is supported by the following country studies.For Germany, we collected a sample of 500 unlisted firms with ultimate owners from Wer gehört zu wem.22 We found that the average control stake is 89.44 percent. In 68 percent of the cases, the largest owner is the sole owner; in the remainder, the largest owner holds an average stake of 67 percent. Families, both domestic and foreign, are the largest ultimate owners of 90.6 percen t of these firms. At the 20 percent level, financial firms control 4.9 percent of unlisted firms, the State 2.67 percent, and cross-holdings 1.83 percent. Thus, both State and widely-held financial firms are insignificant as ultimate owners of unlisted firms. Furthermore, since the State and the largest financial institutions are included in the Wer gehört zu wem database, we were able to identify them and trace their ownership chain. This further reduces the possibility of bias in our sample.For Italy, Bianchi et al. (1998) considered a sample of 1,000 manufacturing firms surveyed by the Bank of Italy. They reported that the largest immediate shareholder of these firms held, on average, a direct stake of 67.69 percent. Among all types of immediate sharehol ders, individuals held 4821 This approach is close to that of Claessens et al. (2000), who regarded as a family any controlling shareholder that was an unlisted firm in a business group.22 We considered firms in alphabetical order, including a firm in our sample if we could trace its ultimate owners. We stopped when our sample numbered 500.8percent of equity, non-financial firms 36.9 percent, financial firms 0.17 percent, the State 4.6 percent, and foreign investors 8 percent. We secured a wider sample of 3,800 unlisted Italian firms with ultimate owners from the AIDA database.23 We found that the State is the largest owner for 0.4 percent of firms, financial institutions for 0.2 percent, and families (domestic and foreign) for 99.4 percent. The average ultimate control stake of the largest controlling shareholder was 70.71 percent.For France, Bloch and Kremp (1998) summarize the ownership structure of a sample of 282,322 (mostly unlisted) firms.24 For firms with more than 500 employees, the largest owner held, on average, 88 percent of the capital. For 56 percent of th e unlisted firms, the largest owner was a family; for the remaining 44 percent it was another corporation, usually an unlisted firm. For the U.K., Goergen and Renneboog (1998) reported that 78 percent of unlisted firms are fully controlled by one shareholder, while the remaining 22 percent have a shareholder who (directly) holds a majority stake.252. 4. Examples[Insert Figure 1 about here]Figure 1 illustrates dual class shares and pyramiding in the Nordström family group of Sweden. All holdings of more than 5 percent of a firm’s shares are listed. All firms shown in Figure 1 have dual class shares: A-shares carry one voting right while B-shares carrying one-tenth of a voting right. Realia has two classes of shares: 2.641 million A-shares and 42.922 million B-shares. A- and B-shares have the same face value, so the total stock capital is 45.563 million shares. A-shares constitute 5.8 percent of stock capital (= 2.641 million / 45.563 million) while B-shares constitute 94.2 percent). A-shares carry 2.641 million votes, while B-shares carry 4.292 million votes (= one-tenth of 42.922 million).23 AIDA is a private database provided by Bureau Van Dijk. The AIDA database provides accounting information for about 130,000 Italian firms and ownership information for 25,314 Italian firms. Fi rms with ownership data in the AIDA database are not ranked in any order. We traced ownership to ultimate owners starting from the first firm listed in the AIDA database using ownership information contained in that database. A firm was included if we could trace its ultimate owners. We stopped when our sample reached 15 percent of firms with ownership data.24 Their results were based on data compiled by the French central bank (Fiben), that is unavailable to the public.25 The figures are based on a sample of 12,600 unlisted firms from the Jordan’s database, which is compiled by a private data vendor, Bureau Van Djik.9Thus, A-shares carry 38.09 percent of the votes (= 2.641 million /(2.641 million + 4.292 million)), while B-shares carry 61.91 percent of votes.Realia has three direct shareholders: Columna Fastigheter, Lingfield Investment and the Eriksson family. Columna owns 1.933 million A-shares and 14.007 million B-shares. Thus, Columna owns 35 percent (= (1.933 million + 14.007 million)/ 45.563 million) of cash flow rights, a nd controls 48.1 percent (= (1.933 million + 1.401 million) / (2.641 million + 4.292 million)) of votes in Realia. Realia’s second largest direct shareholder, Lingfield Investments, owns no A-shares and only 6.259 million B-shares, which comprise 13.7 percent (6.259 million / 45.563 million) of the cash flow rights, and 9 percent (0.6259 / (2.641 million + 4.292 million)) of the control rights. Finally, the Eriksson family owns 0.591 million A-shares and 0.101 million B-shares, comprising 1.5 percent (= (0.591 million + 0.101 million) / 45.563 million) of the ownership rights, and 8.7 percent (= (0.591 million + 0.0101 million) / (2.641 million + 4.292 million)) of the control rights. Through Columna Fastigheter, the Nordström family has sole control of Realia at the 20 percent threshold. However, at the 10 percent threshold, Realia has a second large shareholder, Blockfield Properties.In Relia’s control structure there are two pyramiding chains: Nordström family/Columna/Realia and Blockfield/Columna/Realia.[Insert Figure 2 about here]Figure 2 illustrates the control of Unicem by pyramiding, holdings through multiple control chains, and dual class shares within the Agnelli family group, the largest Italian group. The methodology presented in Figure 1 is used to compute cash flow rights (O) and control rights (C), taking account of dual class shares. Unicem is directly controlled by two major shareholders: Ifi and Ifil. Ifil is controlled by Ifi with a direct stake (O = 7.97 percent; C = 14.6 percent) and an indirect stake (O = 20.55 percent; C = 37.64 percent) through Carfin, a wholly owned, non-financial unlisted firm. Since Carfin is wholly controlled by Ifi, we consider Ifi’s stake in Ifil as a direct holding rather than a pyramid, although we say there is holdings through multiple control chain (see footnote 17). Ifi is controlled by a single major shareholder, Giovanni Agnelli & C. S.p.A. (the Agnelli family). The Agnelli family’s control of Unicem is thus exercised through pyramiding (Ifi-Carfin-Ifil-Unicem), non-voting shares (within Ifi, Ifil,10。

基于Tikhonov正则化的扩展Kaczmarz算法
高蒙;张建军
【期刊名称】《应用数学与计算数学学报》
【年(卷),期】2018(032)004
【摘要】为了更加有效地处理不适定问题,在扩展Kaczmarz算法的思想基础上,提出一种基于Tikhonov正则化的最大残差控制的扩展Kaczmarz算法并证明其收敛性.利用sheep-logan头部图像等进行图像重建实验.数值结果表明,该算法和最大残差控制的扩展Kaczmarz算法(MREK算法)相比,误差更小,图像质量更优.
【总页数】12页(P879-890)
【作者】高蒙;张建军
【作者单位】上海大学理学院,上海200444;上海大学理学院,上海200444
【正文语种】中文
【中图分类】O241.6;O176.3;TP751.1
【相关文献】
1.基于混沌粒子群算法的Tikhonov正则化参数选取 [J], 余瑞艳
2.基于Tikhonov和变差正则化的磁感应断层成像重建算法 [J], 陈玉艳;王旭;吕轶;杨丹
3.基于初至波走时层析成像的Tikhonov正则化与梯度优化算法 [J], 崔岩;王彦飞
4.基于Extrapolation Tikhonov正则化算法的重力数据三维约束反演 [J], 刘银萍;王祝文;杜晓娟;刘菁华;许家姝
5.基于Tikhonov正则化算法的低压电网中电力设备工频电场逆向监测 [J], 李佳;林智炳;吴同金
因版权原因，仅展示原文概要，查看原文内容请购买。

第40卷第9期2023年9月控制理论与应用Control Theory&ApplicationsV ol.40No.9Sep.2023不对称约束多人非零和博弈的自适应评判控制李梦花,王鼎,乔俊飞†(北京工业大学信息学部,北京100124;计算智能与智能系统北京市重点实验室,北京100124;智慧环保北京实验室,北京100124;北京人工智能研究院,北京100124)摘要:本文针对连续时间非线性系统的不对称约束多人非零和博弈问题,建立了一种基于神经网络的自适应评判控制方法.首先,本文提出了一种新颖的非二次型函数来处理不对称约束问题,并且推导出最优控制律和耦合Hamilton-Jacobi方程.值得注意的是,当系统状态为零时,最优控制策略是不为零的,这与以往不同.然后,通过构建单一评判网络来近似每个玩家的最优代价函数,从而获得相关的近似最优控制策略.同时,在评判学习期间发展了一种新的权值更新规则.此外,通过利用Lyapunov理论证明了评判网络权值近似误差和闭环系统状态的稳定性.最后,仿真结果验证了本文所提方法的有效性.关键词:神经网络;自适应评判控制;自适应动态规划;非线性系统;不对称约束;多人非零和博弈引用格式:李梦花,王鼎,乔俊飞.不对称约束多人非零和博弈的自适应评判控制.控制理论与应用,2023,40(9): 1562–1568DOI:10.7641/CTA.2022.20063Adaptive critic control for multi-player non-zero-sum games withasymmetric constraintsLI Meng-hua,WANG Ding,QIAO Jun-fei†(Faculty of Information Technology,Beijing University of Technology,Beijing100124,China;Beijing Key Laboratory of Computational Intelligence and Intelligent System,Beijing100124,China;Beijing Laboratory of Smart Environmental Protection,Beijing100124,China;Beijing Institute of Artificial Intelligence,Beijing100124,China)Abstract:In this paper,an adaptive critic control method based on the neural networks is established for multi-player non-zero-sum games with asymmetric constraints of continuous-time nonlinear systems.First,a novel nonquadratic func-tion is proposed to deal with asymmetric constraints,and then the optimal control laws and the coupled Hamilton-Jacobi equations are derived.It is worth noting that the optimal control strategies do not stay at zero when the system state is zero, which is different from the past.After that,only a critic network is constructed to approximate the optimal cost function for each player,so as to obtain the associated approximate optimal control strategies.Meanwhile,a new weight updating rule is developed during critic learning.In addition,the stability of the weight estimation errors of critic networks and the closed-loop system state is proved by utilizing the Lyapunov method.Finally,simulation results verify the effectiveness of the method proposed in this paper.Key words:neural networks;adaptive critic control;adaptive dynamic programming;nonlinear systems;asymmetric constraints;multi-player non-zero-sum gamesCitation:LI Menghua,WANG Ding,QIAO Junfei.Adaptive critic control for multi-player non-zero-sum games with asymmetric constraints.Control Theory&Applications,2023,40(9):1562–15681引言自适应动态规划(adaptive dynamic programming, ADP)方法由Werbos[1]首先提出,该方法结合了动态规划、神经网络和强化学习,其核心思想是利用函数近似结构来估计最优代价函数,从而获得被控系统的近似最优解.在ADP方法体系中,动态规划蕴含最优收稿日期:2022−01−21;录用日期:2022−11−10.†通信作者.E-mail:***************.cn.本文责任编委:王龙.科技创新2030–“新一代人工智能”重大项目(2021ZD0112302,2021ZD0112301),国家重点研发计划项目(2018YFC1900800–5),北京市自然科学基金项目(JQ19013),国家自然科学基金项目(62222301,61890930–5,62021003)资助.Supported by the National Key Research and Development Program of China(2021ZD0112302,2021ZD0112301,2018YFC1900800–5),the Beijing Natural Science Foundation(JQ19013)and the National Natural Science Foundation of China(62222301,61890930–5,62021003).第9期李梦花等:不对称约束多人非零和博弈的自适应评判控制1563性原理提供理论基础,神经网络作为函数近似结构提供实现手段,强化学习提供学习机制.值得注意的是, ADP方法具有强大的自学习能力,在处理非线性复杂系统的最优控制问题上具有很大的潜力[2–7].此外, ADP作为一种近似求解最优控制问题的新方法,已经成为智能控制与计算智能领域的研究热点.关于ADP的详细理论研究以及相关应用,读者可以参考文献[8–9].本文将基于ADP的动态系统优化控制统称为自适应评判控制.近年来,微分博弈问题在控制领域受到了越来越多的关注.微分博弈为研究多玩家系统的协作、竞争与控制提供了一个标准的数学框架,包括二人零和博弈、多人零和博弈以及多人非零和博弈等.在零和博弈问题中,控制输入试图最小化代价函数而干扰输入试图最大化代价函数.在非零和博弈问题中,每个玩家都独立地选择一个最优控制策略来最小化自己的代价函数.值得注意的是,零和博弈问题已经被广泛研究.在文献[10]中,作者提出了一种改进的ADP方法来求解多输入非线性连续系统的二人零和博弈问题.An等人[11]提出了两种基于积分强化学习的算法来求解连续时间系统的多人零和博弈问题.Ren等人[12]提出了一种新颖的同步脱策方法来处理多人零和博弈问题.然而,关于非零和博弈[13–14]的研究还很少.此外,控制约束在实际应用中也广泛存在.这些约束通常是由执行器的固有物理特性引起的,如气压、电压和温度.因此,为了确保被控系统的性能,受约束的系统需要被考虑.Zhang等人[15]发展了一种新颖的事件采样ADP方法来求解非线性连续约束系统的鲁棒最优控制问题.Huo等人[16]研究了一类非线性约束互联系统的分散事件触发控制问题.Yang和He[17]研究了一类具有不匹配扰动和输入约束的非线性系统事件触发鲁棒镇定问题.这些文献考虑的都是对称约束,而实际应用中,被控系统受到的约束也可能是不对称的[18–20],例如在污水处理过程中,需要通过氧传递系数和内回流量对溶解氧浓度和硝态氮浓度进行控制,而根据实际的运行条件,这两个控制变量就需要被限制在一个不对称约束范围内[20].因此,在控制器设计过程中,不对称约束问题将是笔者研究的一个方向.到目前为止,关于具有控制约束的微分博弈问题,有一些学者取得了相应的研究成果[12,21–23].但可以发现,具有不对称约束的多人非零和博弈问题还没有学者研究.同时,在多人非零和博弈问题中,相关的耦合Hamilton-Jacobi(HJ)方程是很难求解的.因此,本文针对一类连续时间非线性系统的不对称约束多人非零和博弈问题,提出了一种自适应评判控制方法来近似求解耦合HJ方程,从而获得被控系统的近似最优解.本文的主要贡献如下:1)首次将不对称约束应用到连续时间非线性系统的多人非零和博弈问题中;2)提出了一种新颖的非二次型函数来处理不对称约束问题,并且当系统状态为零时,最优控制策略是不为零的,这与以往不同;3)在学习期间,用单一评判网络结构代替了传统的执行–评判网络结构,并且提出了一种新的权值更新规则;4)利用Lyapunov方法证明了评判网络权值近似误差和系统状态的一致最终有界(uniformly ultimately bounded,UUB)稳定性.2问题描述考虑以下具有不对称约束的N–玩家连续时间非线性系统:˙x(t)=f(x(t))+N∑j=1g j(x(t))u j(t),(1)其中:x(t)∈Ω⊂R n是状态向量且x(0)=x0为初始状态,R n代表由所有n-维实向量组成的欧氏空间,Ω是R n的一个紧集;u j(t)∈T j⊂R m为玩家j在时刻t所选择的策略,且T j为T j={[u j1u j2···u jm]T∈R m:u j min u jl u j max, |u j min|=|u j max|,l=1,2,···,m},(2)其中:u jmin∈R和u j max∈R分别代表控制输入分量的最小界和最大界,R表示所有实数集.假设1非线性系统(1)是可控的,并且x=0是被控系统(1)的一个平衡点.此外,∀j∈N,f(x)和g j(x)是未知的Lipschitz函数且f(0)=0,其中集合N={1,2,···,N},N 2是一个正整数.假设2∀j∈N,g j(0)=0,且存在一个正常数b gj使∥g j(x)∥ b gj,其中∥·∥表示在R n上的向量范数或者在R n×m上的矩阵范数,R n×m代表由所有n×m维实矩阵组成的空间.注1假设1–3是自适应评判领域的常用假设,例如文献[6,13,19],是为了保证系统的稳定性以及方便后文中的稳定性证明,其中假设3出现在后文中的第3.2节.定义与每个玩家相关的效用函数为U i(x,U)=x T Q i x+N∑j=1S j(u j),i∈N,(3)其中U={u1,u2,···,u N}并且Q i是一个对称正定矩阵.此外,为了处理不对称约束问题,令S j(u j)为S j(u j)=2αj m∑l=1ujlβjtanh−1(z−βjαj)d z,(4)其中αj和βj分别为αj=u jmax−u j min2,βj=u jmax+u jmin2.(5)因此,与每个玩家相关的代价函数可以表示为J i(x0,U)=∞U i(x,U)dτ,i∈N,(6)1564控制理论与应用第40卷本文希望构建一个Nash均衡U∗={u∗1,u∗2,···,u∗N},来使以下不等式被满足:J i(u∗1,···,u∗i,···,u∗N)J i(u∗1,···,u i,···,u∗N),(7)其中i∈N.为了方便,将J i(x0,U)简写为J i(x0).于是,每个玩家的最优代价函数为J∗i (x0)=minu iJ i(x0,U),i∈N.(8)在本文中,如果一个控制策略集的所有元素都是可容许的,那么这个集合是可容许的.定义1(容许控制[24])如果控制策略u i(x)是连续的,u i(x)可以镇定系统(1),并且J i(x0)是有限的,那么它是集合Ω上关于代价函数(6)的可容许控制律,即u i(x)∈Ψ(Ω),i∈N,其中,Ψ(Ω)是Ω上所有容许控制律的集合.对于任意一个可容许控制律u i(x)∈Ψ(Ω),如果相关代价函数(6)是连续可微的,那么非线性Lyapu-nov方程为0=U i(x,U)+(∇J i(x))T(f(x)+N∑j=1g j(x)u j),(9)其中:i∈N,J i(0)=0,并且∇(·) ∂(·)∂x.根据最优控制理论,耦合HJ方程为0=minU H i(x,U,∇J∗i(x)),i∈N,(10)其中,Hamiltonian函数H i(x,U,∇J∗i(x))为H i(x,U,∇J∗i(x))=U i(x,U)+(∇J∗i (x))T(f(x)+N∑j=1g j(x)u j),(11)进而,由∂H i(x,U,∇J∗i(x))∂u i=0可得出最优控制律为u∗i (x)=−αi tanh(12αig Ti(x)∇J∗i(x))+¯βi,i∈N,(12)其中¯βi=[βiβi···βi]T∈R m.注2根据式(2)和式(5),能推导出βi=0,即¯βi=0,又根据式(12)可知u∗i(0)=0,i∈N.因此,为了保证x=0是系统(1)的平衡点,在假设2中提出了条件∀j∈N,g j(0)=0.将式(12)代入式(10),耦合HJ方程又能表示为(∇J∗i (x))T f(x)+N∑j=1((∇J∗i(x))T g j(x)¯βj)+x T Q i x−N∑j=1((∇J∗i(x))Tαj g j(x)tanh(A j(x)))+N∑j=1S j(−αj tanh(A j(x))+¯βj)=0,i∈N,(13)其中J∗i(0)=0并且A j(x)=12αjg Tj(x)∇J∗j(x).如果已知每个玩家的最优代价函数值,那么相关的最优状态反馈控制律就可以直接获得,也就是说式(13)是可解的.可是,式(13)这种非线性偏微分方程的求解是十分困难的.同时,随着系统维数的增加,存储量和计算量也随之以指数形式增加,也就是平常所说的“维数灾”问题.因此,为了克服这些弱点,在第3部分提出了一种基于神经网络的自适应评判机制,来近似每个玩家的最优代价函数,从而获得相关的近似最优状态反馈控制策略.3自适应评判控制设计3.1神经网络实现本节的核心是构建并训练评判神经网络,以得到训练后的权值,从而获得每个玩家的近似最优代价函数值.首先,根据神经网络的逼近性质[25],可将每个玩家的最优代价函数J∗i(x)在紧集Ω上表示为J∗i(x)=W Tiσi(x)+ξi(x),i∈N,(14)其中:W i∈Rδ是理想权值向量,σi(x)∈Rδ是激活函数,δ是隐含层神经元个数,ξi(x)∈R是重构误差.同时,可得出每个玩家的最优代价函数梯度为∇J∗i(x)=(∇σi(x))T W i+∇ξi(x),i∈N,(15)将式(15)代入式(12),有u∗i(x)=−αi tanh(B i(x)+C i(x))+¯βi,i∈N,(16)其中:B i(x)=12αig Ti(x)(∇σi(x))T W i∈R m,C i(x)=12αig Ti(x)∇ξi(x)∈R m.然后,将式(15)代入式(13),耦合HJ方程变为W Ti∇σi(x)f(x)+(∇ξi(x))T f(x)+x T Q i x+N∑j=1((W Ti∇σi(x)+(∇ξi(x))T)g j(x)¯βj)−N∑j=1(αj W Ti∇σi(x)g j(x)tanh(B j(x)+C j(x)))−N∑j=1(αj(∇ξi(x))T g j(x)tanh(B j(x)+C j(x)))+N∑j=1S j(−αj tanh(B j(x)+C j(x))+¯βj)=0,i∈N.(17)值得注意的是,式(14)中的理想权值向量W i是未知的,也就是说式(16)中的u∗i(x)是不可解的.因此,第9期李梦花等:不对称约束多人非零和博弈的自适应评判控制1565构建如下的评判神经网络:ˆJ∗i (x)=ˆW Tiσi(x),i∈N,(18)来近似每个玩家的最优代价函数,其中ˆW i∈Rδ是估计的权值向量.同时,其梯度为∇ˆJ∗i(x)=(∇σi(x))TˆW i,i∈N.(19)考虑式(19),近似的最优控制律为ˆu∗i(x)=−αi tanh(D i(x))+¯βi,i∈N,(20)其中D i(x)=12αig Ti(x)(∇σi(x))TˆW i.同理,近似的Hamiltonian可以写为ˆHi(x,ˆW i)=ˆW T i ϕi+x T Q i x+N∑j=1(ˆW Ti∇σi(x)g j(x)¯βj)−N ∑j=1(αjˆW Ti∇σi(x)g j(x)tanh(D j(x)))+N∑j=1S j(−αj tanh(D j(x))+¯βj),i∈N,(21)其中ϕi=∇σi(x)f(x).此外,定义误差量e i=ˆH i(x,ˆW i )−H i(x,U∗,∇J∗i(x))=ˆH i(x,ˆW i).为了使e i足够小,需要训练评判网络来使目标函数E i=12e Tie i最小化.在这里,本文采用的训练准则为˙ˆW i =−γi1(1+ϕTiϕi)2(∂E i∂ˆW i)=−γiϕi(1+ϕTiϕi)2e i,i∈N,(22)其中:γi>0是评判网络的学习率,(1+ϕT iϕi)2用于归一化操作.此外,定义评判网络的权值近似误差为˜Wi=W i−ˆW i.因此,有˙˜W i =γiφi1+ϕTiϕie Hi−γiφiφT i˜W i,i∈N,(23)其中:φi=ϕi(1+ϕTiϕi),e Hi=−(∇ξi(x))T f(x)是残差项.3.2稳定性分析本节的核心是通过利用Lyapunov方法讨论评判网络权值近似误差和闭环系统状态的UUB稳定性.这里,给出以下假设:假设3∥∇ξi(x)∥ b∇ξi ,∥∇σi(x)∥ b∇σi,∥e Hi∥ b e Hi,∥W i∥ b W i,其中:b∇ξi,b∇σi,b e Hi,b W i 都是正常数,i∈N.定理1考虑系统(1),如果假设1–3成立,状态反馈控制律由式(20)给出,且评判网络权值通过式(22)进行训练,则评判网络权值近似误差˜W i是UUB 稳定的.证选取如下的Lyapunov函数:L1(t)=N∑i=1(12˜W Ti˜Wi)=N∑i=1L1i(t),(24)计算L1i(t)沿着式(23)的时间导数,即˙L1i(t)=γi˜W Tiφi1+ϕTiϕie Hi−γi˜W TiφiφTi˜Wi,i∈N,(25)利用不等式¯X T¯Y12∥¯X∥2+12∥¯Y∥2(注:¯X和¯Y都是具有合适维数的向量),并且考虑1+ϕTiϕi 1,能得到˙L1i(t)γi2(∥φTi˜Wi∥2+∥e Hi∥2)−γi˜W TiφiφTi˜Wi=−γi2˜W TiφiφTi˜Wi+γi2∥e Hi∥2,i∈N.(26)根据假设3,有˙L1i(t) −γi2λmin(φiφTi)∥˜W i∥2+γi2b2e Hi,i∈N,(27)其中λmin(·)表示矩阵的最小特征值.因此,当不等式∥˜W i∥>√b2e Hiλmin(φiφTi),i∈N(28)成立时,有˙L1i(t)<0.根据标准的Lyapunov定理[26],可知评判网络权值近似误差˜W i是UUB稳定的.证毕.定理2考虑系统(1),如果假设1–3成立,状态反馈控制律由式(20)给出,且评判网络权值通过式(22)进行训练,则系统状态x(t)是UUB稳定的.证选取如下的Lyapunov函数:L2i(t)=J∗i(x),i∈N.(29)计算L2i(t)沿着系统˙x=f(x)+N∑j=1g j(x)ˆu∗j的时间导数,即˙L2i(t)=(∇J∗i(x))T(f(x)+N∑j=1g j(x)ˆu∗j)=(∇J∗i(x))T(f(x)+N∑j=1g j(x)u∗j)+N∑j=1((∇J∗i(x))T g j(x)(ˆu∗j−u∗j)),i∈N.(30)考虑式(13),有˙L2i(t)=−x T Q i x−N∑j=1S j(u∗j)+N∑j=1((∇J∗i(x))T g j(x)(ˆu∗j−u∗j))Σi,i∈N,(31)1566控制理论与应用第40卷利用不等式¯XT ¯Y 12∥¯X ∥2+12∥¯Y ∥2,并且考虑式(15)–(16)(20),可得Σi 12N ∑j =1∥−αj tanh (D j (x ))+αj tanh (F j (x ))∥2+12N ∑j =1∥g Tj (x )((∇σi (x ))T W i +∇ξi (x ))∥2,i ∈N ,(32)其中F j (x )=B j (x )+C j (x ).然后,利用不等式∥¯X+¯Y∥2 2∥¯X ∥2+2∥¯Y ∥2,有Σi N ∑j =1(∥αj tanh (D j (x ))∥2+∥αj tanh (F j (x ))∥2)+N ∑j =1∥g Tj (x )(∇σi (x ))T W i ∥2+N ∑j =1∥g T j (x )∇ξi (x )∥2,i ∈N ,(33)其中D j (x )∈R m ,F j (x )∈R m 分别被表示为[D j 1(x )D j 2(x )···D jm (x )]T 和[F j 1(x )F j 2(x )···F jm (x )]T .易知,∀θ∈R ,tanh 2θ 1.因此,有∥tanh (D j (x ))∥2=m ∑l =1tanh 2(D jl (x )) m,(34)∥tanh (F j (x ))∥2=m ∑l =1tanh 2(F jl (x )) m.(35)同时,根据假设2–3,有Σi N ∑j =1(2α2j m +b 2g j b 2∇σi b 2W i +b 2g j b 2∇ξi ),i ∈N ,(36)根据式(2)(4)–(5),可知S j (u ∗j ) 0.于是,有˙L2i (t ) −λmin (Q i )∥x ∥2+ϖi ,i ∈N ,(37)其中ϖi =N ∑j =1(2α2j m +b 2g j b 2∇σi b 2W i +b 2g j b 2∇ξi ).因此,根据式(37)可知,当不等式∥x ∥>√ϖiλmin (Q i )成立时,有˙L2i (t )<0.即,如果x (t )满足下列不等式:∥x ∥>max {√ϖ1λmin (Q 1),···,√ϖNλmin (Q N )},(38)则,∀i ∈N ,都有˙L 2i (t )<0.同理,可得闭环系统状态x (t )也是UUB 稳定的.证毕.4仿真结果考虑如下的3–玩家连续时间非线性系统:˙x =[−1.2x 1+1.5x 2sin x 20.5x 1−x 2]+[01.5sin x 1cos x 1]u 1(x )+[1.2sin x 1cos x 2]u 2(x )+[01.1sin x 2]u 3(x ),(39)其中:x (t )=[x 1x 2]T ∈R 2是状态向量,u 1(x )∈T 1={u 1∈R :−1 u 1 2},u 2(x )∈T 2={u 2∈R :−0.2 u 2 1}和u 3(x )∈T 3={u 3∈R :−0.4 u 3 0.8}是控制输入.令Q 1=2I 2,Q 2=1.8I 2,Q 3=0.3I 2,其中I 2代表2×2维单位矩阵.同时,根据式(5)可知,α1=1.5,β1=0.5,α2=0.6,β2=0.4,α3=0.6,β3=0.2.因此,与每个玩家相关的代价函数可以表示为J i (x 0)= ∞0(x TQ i x +3∑j =1S j (u j ))d τ,i =1,2,3,(40)其中S j (u j )=2αju jβj tanh −1(z −βjαj)d z =2αj (u j −βj )tanh −1(u j −βjαj)+α2j ln (1−(u j −βj )2α2j).(41)然后,本文针对系统(39)构建3个评判神经网络,每个玩家的评判神经网络权值分别为ˆW1=[ˆW 11ˆW 12ˆW13]T ,ˆW 2=[ˆW 21ˆW 22ˆW 23]T ,ˆW 3=[ˆW 31ˆW 32ˆW33]T ,激活函数被定义为σ1(x )=σ2(x )=σ3(x )=[x 21x 1x 2x 22]T,且隐含层神经元个数为δ=3.此外,系统初始状态取x 0=[0.5−0.5]T ,每个评判神经网络的学习率分别为γ1=1.5,γ2=0.8,γ3=0.2,且每个评判神经网络的初始权值都在0和2之间选取.最后,引入探测噪声η(t )=sin 2(−1.2t )cos(0.5t )+cos(2.4t )sin 3(2.4t )+sin 5t +sin 2(1.12t )+sin 2t ×cos t +sin 2(2t )cos(0.1t ),使得系统满足持续激励条件.执行学习过程,本文发现每个玩家的评判神经网络权值分别收敛于[6.90912.99046.6961]T ,[4.89012.23475.2062]T ,[1.79450.33212.4583]T .在60个时间步之后去掉探测噪声,每个玩家的评判网络权值收敛过程如图1–3所示.然后,将训练好的权值代入式(20),能得到每个玩家的近似最优控制律,将其应用到系统(39),经过10个时间步之后,得到的状态轨迹和控制轨迹分别如图4–5所示.由图4可知,系统状态最终收敛到了平衡点.由图5可知,每个玩家的控制轨迹都没有超出预定的边界,并且可以观察到u 1,u 2和u 3分别收敛于0.5,0.4和0.2.综上所述,仿真结果验证了所提方法的有效性.第9期李梦花等:不对称约束多人非零和博弈的自适应评判控制1567䇴 㖁㔌U / s图1玩家1的评判网络权值收敛过程Fig.1Convergence process of the critic network weights forplayer1䇴 㖁㔌U / s图2玩家2的评判网络权值收敛过程Fig.2Convergence process of the critic network weights forplayer2﹣䇴 㖁㔌U / s图3玩家3的评判网络权值收敛过程Fig.3Convergence process of the critic network weights forplayer 35结论本文首次将不对称约束应用到连续时间非线性系统的多人非零和博弈问题中.首先,获得了最优状态反馈控制律和耦合HJ 方程,并且为了解决不对称约束问题,建立了一种新的非二次型函数.值得注意的是,当系统状态为零时,最优控制策略是不为零的.其次,由于耦合HJ 方程不易求解,提出了一种基于神经网络的自适应评判算法来近似每个玩家的最优代价函数,从而获得相关的近似最优控制律.在实现过程中,用单一评判网络结构代替了经典的执行–评判结构,并且建立了一种新的权值更新规则.然后,利用Lyap-unov 理论讨论了评判网络权值近似误差和系统状态的UUB 稳定性.最后,仿真结果验证了所提算法的可行性.在未来的工作中,会考虑将事件驱动机制引入到连续时间非线性系统的不对称约束多人非零和博弈问题中,并且将该研究内容应用到污水处理系统中也是笔者的一个重点研究方向.﹣0.5﹣0.4﹣0.3﹣0.2﹣0.10.00.10.20.00.10.20.30.40.5(U )Y 1(U )Y 2图4系统(39)的状态轨迹Fig.4State trajectory of the system (39)0.00.51.01.52.00.00.20.40.60.81.01.200.012345678910﹣0.40.4﹣0.20.2(U )V 3(U )V 2(U )V 1U / s 012345678910U / s 012345678910U / s (c)(b)(a)(U )V 1(U )V 2(U )V 3图5系统(39)的控制轨迹Fig.5Control trajectories of the system (39)1568控制理论与应用第40卷参考文献:[1]WERBOS P J.Beyond regression:New tools for prediction andanalysis in the behavioral sciences.Cambridge:Harvard Universi-ty,1974.[2]HONG Chengwen,FU Yue.Nonlinear robust approximate optimaltracking control based on adaptive dynamic programming.Control Theory&Applications,2018,35(9):1285–1292.(洪成文,富月.基于自适应动态规划的非线性鲁棒近似最优跟踪控制.控制理论与应用,2018,35(9):1285–1292.)[3]CUI Lili,ZHANG Yong,ZHANG Xin.Event-triggered adaptive dy-namic programming algorithm for the nonlinear zero-sum differential games.Control Theory&Applications,2018,35(5):610–618.(崔黎黎,张勇,张欣.非线性零和微分对策的事件触发自适应动态规划算法.控制理论与应用,2018,35(5):610–618.)[4]WANG D,HA M,ZHAO M.The intelligent critic framework foradvanced optimal control.Artificial Intelligence Review,2022,55(1): 1–22.[5]WANG D,QIAO J,CHENG L.An approximate neuro-optimal solu-tion of discounted guaranteed cost control design.IEEE Transactions on Cybernetics,2022,52(1):77–86.[6]YANG X,HE H.Adaptive dynamic programming for decentralizedstabilization of uncertain nonlinear large-scale systems with mis-matched interconnections.IEEE Transactions on Systems,Man,and Cybernetics:Systems,2020,50(8):2870–2882.[7]ZHAO B,LIU D.Event-triggered decentralized tracking control ofmodular reconfigurable robots through adaptive dynamic program-ming.IEEE Transactions on Industrial Electronics,2020,67(4): 3054–3064.[8]WANG Ding.Research progress on learning-based robust adaptivecritic control.Acta Automatica Sinica,2019,45(6):1037–1049.(王鼎.基于学习的鲁棒自适应评判控制研究进展.自动化学报, 2019,45(6):1037–1049.)[9]ZHANG Huaguang,ZHANG Xin,LUO Yanhong,et al.An overviewof research on adaptive dynamic programming.Acta Automatica Sini-ca,2013,39(4):303–311.(张化光,张欣,罗艳红,等.自适应动态规划综述.自动化学报, 2013,39(4):303–311.)[10]L¨U Yongfeng,TIAN Jianyan,JIAN Long,et al.Approximate-dynamic-programming H∞controls for multi-input nonlinear sys-tem.Control Theory&Applications,2021,38(10):1662–1670.(吕永峰,田建艳,菅垄,等.非线性多输入系统的近似动态规划H∞控制.控制理论与应用,2021,38(10):1662–1670.)[11]AN P,LIU M,WAN Y,et al.Multi-player H∞differential gameusing on-policy and off-policy reinforcement learning.The16th In-ternational Conference on Control and Automation.Electr Network: IEEE,2020,10:1137–1142.[12]REN H,ZHANG H,MU Y,et al.Off-policy synchronous iterationIRL method for multi-player zero-sum games with input constraints.Neurocomputing,2020,378:413–421.[13]LIU D,LI H,WANG D.Online synchronous approximate optimallearning algorithm for multiplayer nonzero-sum games with unknown dynamics.IEEE Transactions on Systems,Man,and Cybernetics: Systems,2014,44(8):1015–1027.[14]V AMVOUDAKIS K G,LEWIS F L.Non-zero sum games:Onlinelearning solution of coupled Hamilton-Jacobi and coupled Riccati equations.IEEE International Symposium on Intelligent Control.Denver,CO,USA:IEEE,2011,9:171–178.[15]ZHANG H,ZHANG K,XIAO G,et al.Robust optimal controlscheme for unknown constrained-input nonlinear systems via a plug-n-play event-sampled critic-only algorithm.IEEE Transactions on Systems,Man,and Cybernetics:Systems,2020,50(9):3169–3180.[16]HUO X,KARIMI H R,ZHAO X,et al.Adaptive-critic design fordecentralized event-triggered control of constrained nonlinear inter-connected systems within an identifier-critic framework.IEEE Trans-actions on Cybernetics,2022,52(8):7478–7491.[17]YANG X,HE H.Event-triggered robust stabilization of nonlin-ear input-constrained systems using single network adaptive critic designs.IEEE Transactions on Systems,Man,and Cybernetics:Sys-tems,2020,50(9):3145–3157.[18]WANG L,CHEN C L P.Reduced-order observer-based dynamicevent-triggered adaptive NN control for stochastic nonlinear systems subject to unknown input saturation.IEEE Transactions on Neural Networks and Learning Systems,2021,32(4):1678–1690.[19]YANG X,ZHU Y,DONG N,et al.Decentralized event-driven con-strained control using adaptive critic designs.IEEE Transactions on Neural Networks and Learning Systems,2022,33(10):5830–5844.[20]WANG D,ZHAO M,QIAO J.Intelligent optimal tracking withasymmetric constraints of a nonlinear wastewater treatment system.International Journal of Robust and Nonlinear Control,2021,31(14): 6773–6787.[21]LI M,WANG D,QIAO J,et al.Neural-network-based self-learningdisturbance rejection design for continuous-time nonlinear con-strained systems.Proceedings of the40th Chinese Control Confer-ence.Shanghai,China:IEEE,2021,7:2179–2184.[22]SU H,ZHANG H,JIANG H,et al.Decentralized event-triggeredadaptive control of discrete-time nonzero-sum games over wireless sensor-actuator networks with input constraints.IEEE Transactions on Neural Networks and Learning Systems,2020,31(10):4254–4266.[23]YANG X,HE H.Event-driven H∞-constrained control using adap-tive critic learning.IEEE Transactions on Cybernetics,2021,51(10): 4860–4872.[24]ABU-KHALAF M,LEWIS F L.Nearly optimal control laws for non-linear systems with saturating actuators using a neural network HJB approach.Automatica,2005,41(5):779–791.[25]HORNIK K,STINCHCOMBE M,WHITE H.Universal approxima-tion of an unknown mapping and its derivatives using multilayer feed-forward networks.Neural Networks,1990,3(5):551–560.[26]LEWIS F L,JAGANNATHAN S,YESILDIREK A.Neural NetworkControl of Robot Manipulators and Nonlinear Systems.London:Tay-lor&Francis,1999.作者简介:李梦花博士研究生,目前研究方向为自适应动态规划、智能控制,E-mail:*********************;王鼎教授,博士生导师,目前研究方向为智能控制、强化学习,E-mail:*****************.cn;乔俊飞教授,博士生导师,目前研究方向为智能计算、智能优化控制,E-mail:***************.cn.。

Name _______________________________________ Period ________ ©2010 Secondary Solutions-79- Frankenstein Literature Guide Purchase of this product entitles teacher, school, district, or similar institution, authorized use and reproduction of these materials, in wholeor part, for one teacher in one classroom. Sharing, storing, copying, or posting on the Internet for free or sale, is illegal.Chapters Eighteen – TwentyNote-Taking and Summarizing Directions: For Chapters Eighteen through Twenty, fill in the chart with the necessary information. (Note: Except when writing the summary, you do not need to write in complete sentences.) Chapter EighteenSettingCharactersSummary ofthe ChapterPrediction of Coming EventsChapter NineteenSettingCharactersSummary ofthe ChapterPrediction of Coming EventsChapter TwentySettingCharactersSummary ofthe ChapterPrediction of Coming EventsName _______________________________________ Period ________ ©2010 Secondary Solutions -80- Frankenstein Literature GuidePurchase of this product entitles teacher, school, district, or similar institution, authorized use and reproduction of these materials, in wholeor part, for one teacher in one classroom. Sharing, storing, copying, or posting on the Internet for free or sale, is illegal. Chapters Eighteen – TwentyComprehension Check Directions: To help you understand all aspects of the novel, respond to the following as they relate to Chapters Eighteen through Twenty. Write your responses on a separate piece of paper using complete sentences.Chapter Eighteen1. Tell why Alphonse Frankenstein thinks Victor is depressed.2. Summarize why Victor feels that he cannot marry Elizabeth at this time.3. Demonstrate how Victor manipulates his father so he has the time and liberty to create a female creature.4. Analyze why Victor feels that his family will be safer if he leaves Switzerland.5. Generalize how the setting and scenery of his journey finally affect Victor’s mind and spirits.6. Assess how Victor feels about Clerval’s friendship and companionship.Chapter Nineteen1. Quote the passage(s) that tells how Victor views his life in regards to the monster and his demand.2. While Clerval and Victor are in London, how does Clerval occupy his time?3. Describe how Victor views the process of creating a female creature.4. To what is Shelley alluding in Victor’s assertion: “But I am a blasted tree; the bolt has entered my soul; and I felt then that I should survive to exhibit what I shall soon cease to be—a miserable spectacle of wrecked humanity, pitiable to others, and intolerable to myself”? What does Victor mean?5. Detail why Victor cannot bear to hear Henry speak of Chamounix.6. Explain why Victor sometimes fears for Henry’s life while on their journey.7. What does Victor ask of Henry? What does Victor wish to do on his own?8. Formulate how Victor’s selection of the Orkney Islands mirrors his feelings about the task he plans to complete there.9. Judge Victor’s emotional state as he begins to create a female monster.Chapter Twenty1. What are Victor’s concerns about creating another monster?2. Explain why Victor destroys the creature he is forming.3. Examine how the creature conveys the power he believes he has over Victor.4. Infer what the monster means when he tells Victor, “I shall be with you on your wedding-night.”5. Generalize how Victor feels after the monster threatens him.6. Evaluate Victor’s decision not to create a female monster. Do you think he is making the right decision? Why or why not?7. What does Victor do with the remains of the female creature?8. Describe what Victor does when he becomes tired while sailing.9. Predict why the people in the village think Victor is responsible for the death of a gentleman in the town.。

自由民主与选举民主作者:邰浴日一个自诩为民主政体的国家究竟应当具备哪些核心特征？是不是只要具备了一个多党竞争的选举制度就可以被称为民主制度？如果不是，那么除了选举之外，民主制度还需要具备哪些特征？这是本文试图予以回答的问题。

这篇论文将首先分析民主制度的内涵，并进而探寻民主制度背后的自由主义原则；在第二部分，本文将以美国的宪政制度设计为例来分析自由民主制度所应具备的一些核心要素；本文的第三部分将以一些在第三波民主化浪潮中的国家为例，指出在缺少了一定形式的制度安排的情况下，民主制度并不能有效平稳地运行，亦达不到保障公民权利的目的。

由此我们将提出对于自由民主与选举民主的区分，并对两者的优劣进行对比分析；最后我们将得出结论，即仅仅建立选举民主的制度是远远不够的，一个民主政体的题中应有之义，应当是努力追求建立一个完备的自由民主制度。

一、民主的含义本文的主题是讨论民主制度，那么首先要问的是，民主的定义是什么？著名经济学家熊彼特在其名著《资本主义、社会主义与民主》一书中提出，一个现代民族国家，如果其最强有力的决策者中多数是通过公平、诚实、定期的选举产生的，而且在这样的选举中候选人可以自由地竞争选票，并且实际上每个成年公民都有投票权，那么，这个国家就有了民主政体。

（Schumpeter， 1942，引自Huntington， 1997： 6）如亨廷顿所指出的那样，熊彼特的这一对于民主的程序性定义受到了后世普遍的关注和讨论，如今已得到了在这一领域从事研究的学者的公认。

（Huntington， 1997： 6-7）如我们所知，民主与自由是紧密相关的，事实上，民主制度所依据的理论基础，便来源于古典的自由主义理论。

为了进一步探寻民主制度的内涵与基础，我们不得不对古典自由主义理论的基本主张予以适当的关注。

美国著名政治学者高于斯教授在他的著作中指出了古典自由主义的四个主要特征：首先，古典自由主义将宽容视作人类社会的首要美德，赋予其极高的正面价值；其次，古典自由主义将某种特定的人类自由赋予了特殊的重要性；第三，（古典）自由主义者们都普遍信奉个人主义；最后，古典自由主义以一种对于无限制的集中、专断的权力的担忧与警惕为特征，因此，对于这种权力的限制便一直成为自由主义政治的一个主要目标（Geuss， 2005：14）。

Aging of PopulationLeonid A. Gavrilov and Patrick HeuvelineThis is a manuscript of our article in The Encyclopedia of Population. New York, Macmillan Reference USA, 2003.[Note: This original manuscript is slightly different from the final publication because of small editorial changes.]Reference to the published article:Gavrilov L.A., Heuveline P.“Aging of Population.”In: Paul Demeny and Geoffrey McNicoll (Eds.)The Encyclopedia of Population. New York, Macmillan Reference USA, 2003Available at:/servlet/ItemDetailServlet?region=9&imprint=000&titleCode=M333&ty pe=4&id=174029Aging of population(also known as demographic aging, and population aging) is a summary term for shifts in the age distribution (i.e., age structure) of a population toward older ages. A direct consequence of the ongoing global fertility transition (decline) and of mortality decline at older ages, population aging is expected to be among the most prominent global demographic trends of the 21st century. Population aging is progressing rapidly in many industrialized countries, but those developing countries whose fertility declines began relatively early also are experiencing rapid increases in their proportion of elderly people. This pattern is expected to continue over the next few decades, eventually affecting the entire world. Population aging has many important socio-economic and health consequences, including the increase in the old-age dependency ratio. Itpresents challenges for public health (concerns over possible bankruptcy of Medicare and related programs) as well as for economic development (shrinking and aging of labor force, possible bankruptcy of social security systems).Defining and measuring population agingAs the study of population aging is often driven by a concern over its burdening of retirement systems, the aging of population is often measured by increases in the percentage of elderly people of retirement ages. The definition of retirement ages may vary but a typical cutoff is 65 years, and nowadays a society is considered relatively old when the fraction of the population aged 65 and over exceeds 8-10%. By this standard, the percentage of elderly people in the United States stood at 12.6% in 2000, compared with only 4.1% in 1900 and a projected increase to 20% by the year 2030.A related measure of population aging is the elderly dependency ratio (EDR): the number of individuals of retirement ages compared to the number of those of working ages. For convenience, working ages may be assumed to start at age 15, although increasing proportions of individuals pursue their education beyond that age and remain, meanwhile, financially dependent, either on the state or, increasingly, on their parents or bank managers. The ratio of the elderly dependent population to the economically active (working) population is also known as old-age dependency ratio, age-dependency ratio or elderly dependency burden and is used to assess intergenerational transfers, taxation policies, and saving behavior.Another indicator of the age structure is the aging index (sometimes referred to as the elder-child ratio), defined as the number of people aged 65 and over per 100 youths under age 15. In 2000, only a few countries (Germany, Greece, Italy, Bulgaria, and Japan) had more elderly than youth (aging index above 100). By 2030, however, the aging index is projected to exceed 100 in all developed countries, and the index of several European countries and Japan are even expected to exceed 200. To date, aging indexes are much lower in developing countries than in the developed world, but the proportional rise in the aging index in developing countries is expected to be greater than in developed countries.These indicators of population aging are mere head-count ratios (HCR), that is, they simply relate the number of individuals in large age categories.These indicators fail to take into account the age distribution within these large categories, in particular among the elderly. When the fertility and mortality trends responsible for population aging have been fairly regular over time, the population growth is positively correlated with age (i.e., the oldest age groups are growing fastest). This implies that if the proportion of the population over age 65 is increasing, within that 65-and-over population the proportion over, say, age 80 is also increasing. As health, financial situation, and consumption patterns may vary greatly between 65 year-olds and 80 year-olds, simple ratios conceal important heterogeneity in the elderly population. Increasingly, attention is paid to the "oldest olds" (typically age 80 and over). A long-time subject of curiosity, the number of centenarians is growing even faster. Estimated at 180,000 worldwide in 2000, it could reach 1 million by 2030 (United Nations 2001).The second class of indicators for population aging is the group of statistical measures of location (median, mean and modal ages of population). The median age -- the age at which exactly half the population is older and another half is younger -- is perhaps the most widely used indicator. For the year 2000, the median age in the United States was 36 years, a typical age for most developed countries and twice the median age for Africa (United Nations 2001). Because it is more sensitive to changes at the right-hand tail of the age distribution (i.e., the oldest old ages), the mean age of population might in fact be preferred to the median age to study the dynamics of population aging.Since population aging refers to changes in the entire age distribution, any single indicator might appear insufficient to measure it. The age distribution of population is often very irregular, reflecting the scars of the past events (wars, depression etc.), and it cannot be described just by one number without significant loss of information. Were the age distribution to change in a very irregular fashion over the age range, for instance, much information would be lost by a single-index summary. Therefore, perhaps the most adequate approach to study population aging is to explore the age distribution through a set of percentiles, or graphically by analyzing the population pyramids. Demographers commonly use population pyramids to describe both age and sex distributions of populations. Youthful populations are represented by pyramids with a broad base of young children and a narrow apex of older people, while older populations are characterized bymore uniform numbers of people in the age categories.Figures 1-5 About HereDemographic determinants of population agingTo understand the demographic factors that cause population aging, demographers often refer to stable populations (Preston et al. 2001). This population model assumes that age-specific fertility and mortality rates remain constant over time, and this results in a population with an age distribution that stabilizes and eventually becomes time invariant as well. Conversely, this theoretical model suggests that any change in age structure, and population aging in particular, can only be caused by changes in fertility and mortality rates. The influence of changes in fertility rates on population aging is perhaps less intuitive than that of mortality rates. Everything else constant, however, a fertility decline reduces the size of the most recent birth cohorts relative to the previous birth cohorts, hence reducing the size of the youngest age groups relative to that of the older ones.The effects of changes in mortality rates on population aging appear more intuitive, but are in fact more ambiguous. If increases in the human life span are correctly linked to population aging, reductions in mortality rates do not necessarily contribute to population aging. More specifically, mortality declines among infants, children and persons younger than the population mean age tend to lower the population mean age. A moment of thought suggests that indeed a reduction of neonatal mortality (i.e., death in the first month of life) adds individual at age 0 and should lead to the same partial alleviation of population aging as an increase in childbearing.Population aging is thus related to the demographic transition, that is the processes that lead a society from a demographic regime characterized by high rates of fertility and mortality to another one with lower fertility and mortality rates. In the course of this transition, the age structure is subjected to different influences. In the typical sequence, the transition begins with successes in preventing infectious and parasitic diseases that benefit infants and young children most. The resulting improvement in life expectancy at birth occurs while fertility tends to remain unchanged, thereby producing large birth cohorts and an expanding proportion of children relative to adults. Other things being equal, this initial decline in mortality generates a younger population age structure.After initial and sometimes very rapid gains in infant and child mortality have been achieved, further mortality declines increasingly benefit older ages and are eventually accompanied by fertility declines. Both changes contribute to reverse the early effect of mortality decline on the age structure, and this synergy is known as the double aging process. This corresponds to the experience of most developed countries today, but further decomposition suggest that their history of declining mortality is the dominant factor in current aging (Preston, Himes and Eggers 1989). Mortality declines continue in these countries and the decrease in mortality rates among the oldest-old (85+ years) has actually accelerated since the 1950s (Gavrilov, Gavrilova, 1991). This latest phase of mortality decline, which is concentrated in the older age groups, is becoming an important determinant of population aging, particularly among women.The rate of population aging may also be modulated by migration. Immigration usually slows down population aging (in Canada and Europe, for example), because immigrants tend to be younger and have more children. On the other hand, emigration of working-age adults accelerates population aging, as it is observed now in some Caribbean nations. Population aging in these countries is also accelerated by immigration of elderly retirees from other countries, and return migration of former emigrants who are above the average population age. Some demographers expect that migration will have a more prominent role in population aging in the future, particularly in low-fertility countries with stable or declining population size. The effects of migration on population aging are usually stronger in smaller populations, because of higher relative weight (proportion) of migrants in such populations.Dynamics of population agingThe current level and pace of population aging vary widely by geographic region, and usually within regions as well, but virtually all nations are now experiencing growth in their numbers of elderly residents (for selected regions and countries, see Table 1). The percentage of world population aged 65 and over only increased from 5.2% in 1950 to 6.9% in 2000. In Europe, however, the proportion is 14.7% in 2000. For a long time, the highest proportions where found in Northern Europe (e.g., 10.3% in Sweden in 1950), but had moved South by 2000 (18.1% in Italy). The proportions of elderly arelower outside of Europe with the notable exception of Japan where it increased from 4.9% in 1950 to 17.2% in 2000. The age structure of the United States continues to be marked by the large birth cohorts of the baby boom (people born from 1946 through 1964), not yet aged 65. The proportion of the elderly population in the U.S., 12.3% in 2000, hence remains low compared to the developed-country standards. .Table 1 About HerePopulation aging has the following notable features:(1) The most rapid growth occurs in the oldest age groups – the oldest-old (80+ or 85+ years) and centenarians (100+ years) in particular. In other words, population aging is becoming “deeper” with preferential accumulation of particularly old and frail people.(2) Population aging is particularly rapid among women, resulting in “feminization” of population aging (because of lower mortality rates among women). For example, in the United States, there were 20.6 million older women and 14.4 million older men in 2000, or a sex ratio of 143 women for every 100 men. The female to male ratio increases with age reaching 245 for persons 85 and over.(3) Another consequence of lower female mortality is the fact that almost half of older women (45%) in 2000 were widows, thus living without spousal support.(4) Population aging also causes changes in living arrangements resulting in increasing number of older people living alone (about 30% of all non-institutionalized older persons in 2000 lived alone in the United States).(5) Since older persons have usually lower income and a higher proportion of them are living below the poverty line, population aging is associated with poverty, particularly in developing countries.Projections of population aging in the 21st centuryFuture population aging will depend on future demographic trends, but most demographers agree that the fertility and mortality changes that would be required to reverse population aging in the coming decades are very unlikely. According to current population forecasts, population aging in the first half of this century should exceed that of the second half of the 20thcentury. For the world as a whole, the elderly will grow from 6.9% of the population in 2000 to a projected 19.3% in 2050 (Table 1). In other words, the world average should then be higher than the current world record. All regions are expected to see an increase, although it should be milder in some regions, such as Africa where the projected increase is from 3.3% in 2000 to 6.9% in 2050. But in Latin America and the Caribbean, the increase should be from 5.4% in 2000 to 16.9% in 2050, higher than the current European average. The increase should be even more spectacular in China: from 6.9% in 2000 to 22.7% in 2050.If population aging is thus far from limited to the most developed regions, the countries of these regions will likely continue to experience the highest proportions ever known. The forecasts suggest 29.2% of elderly in the European population as a whole, but more than 30% in a number of specific European countries, and perhaps as much as 36.4% in Japan. Again, the forecasted increase from 12.3% in 2000 to 21.1% in 2050 appears less dramatic in the U.S. than in other most developed countries.There is of course some uncertainty with any forecast, but it is important to note that previous population forecasts underestimated rather than overstated the current pace of population aging. Before the 1980s the process of population aging was considered as an exclusive consequence of fertility decline and it was predicted that the pace of population aging would decrease after stabilization of fertility rates at some low levels. Rapid decline in old-age mortality observed in developed countries in the last decades of the 20th century significantly accelerated population aging. Now the old-age mortality trends are becoming the key demographic component in projecting the size and composition of the world's future elderly population. Current and future uncertainties about changing mortality may produce widely divergent projections of the size of tomorrow's elderly population. For example, the U.S. Census Bureau's middle-mortality series projection suggests that there will be 14.3 million people aged 85 and over in the year 2040, while the low-mortality (i.e., high life expectancy) series implies 16.8 million. Alternative projections, using assumptions of lower death rates and higher life expectancies, have produced estimates from 23.5 to 54 million people aged 85 and over in 2040 in the United States (see Kinsella, Velkoff, 2001).Social and economic implications of population agingWhile population aging represents, in one sense, a success story for mankind (massive survival to old ages has become possible), it also poses profound challenges to public institutions that must adapt to a changing age structure.The first challenge is associated with dramatic increase in the older retired population relative to the shrinking population of working ages, which creates social and political pressures on social support systems. In most developed countries, rapid population aging places a strong pressure on social security programs. For example, the U.S. social security system may face a profound crisis if no radical modifications are enacted. Cuts in benefits, tax increases, massive borrowing, lower cost-of-living adjustments, later retirement ages, or a combination of these elements are now discussed as the possible painful policies, which may become necessary in order to sustain the pay-as-you-go public retirement programs such as Medicare and Social Security.Population aging is also a great challenge for the health care systems. As nations age, the prevalence of disability, frailty, and chronic diseases (Alzheimer’s disease, cancer, cardiovascular and cerebrovascular diseases, etc.) is expected to increase dramatically. Some experts raise concerns that the mankind may become a “global nursing home” (Eberstadt, 1997).The aging of the population is indeed a global phenomenon that requires international coordination of national and local actions. The United Nations and other international organizations developed recommendations intended to mitigate the adverse consequences of population aging. These recommendations include reorganization of social security systems, changes in labor, immigration and family policies, promotion active and healthy life styles, and more cooperation between the governments in resolving socioeconomic and political problems posed by population aging.On the positive side, the health status of older people of a given age is improving over time now, because more recent generations have a lower disease load. Older people can live vigorous and active lives until a much later age than in the past and if they're encouraged to be productive, they can be economic contributors as well. Also the possibility should not be excluded that current intensive biomedical anti-aging studies may help to extend the healthy and productive period of human life in the future (de Greyet al., 2002).Word Count: 2,793BIBLIOGRAPHYAdministration on Aging. 2001. A Profile of Older Americans: 2001. U.S.Department of Health and Human Services.De Grey, Aubrey D. N., Leonid Gavrilov, S. Jay Olshansky, L. Stephen Coles, Richard G. Cutler, Michael Fossel, and S. MitchellHarman. 2002. “Antiaging technology and pseudoscience.” Science, 296: 656-656.Eberstadt, N. 1997. “World population implosion?” Public Interest, 129: 3-22.Gavrilov, Leonid A., and Natalia S. Gavrilova. 1991. The Biology of Life Span: A Quantitative Approach. NY, etc.: Harwood Academic Publ.. Kinsella, Kevin, and Victoria A. Velkoff. 2001. An Aging World: 2001. U. S.Census Bureau, Series P95/01-1, Washington, DC: U.S. GovernmentPrinting Office.Lutz, Wolfgang, Warren Sanderson, and Sergei Scherbow. 2001. “The end of world population growth.” Nature 412: 543-545.Preston, Samuel H., Christine Himes, and MitchellEggers. 1989. “Demographic conditions responsible for populationaging.” Demography 26: 691-704.Preston Samuel H., Patrick Heuveline, and Michel Guillot. 2001.Demography. Measuring and Modeling Population Processes. Oxford: Blackwell.United Nations 2001. World population prospects: the 2000 revision. New York: United Nations.Table 1. Dynamics of Population Aging in the Modern World Observed and Forecasted Percentages of the Elderly (65+ years) in Selected Areas, Regions, and Countries of the World: 1950, 2000 and 2050.Major Area, region and country 1950 20005.2%6.9%WorldAfrica 3.2% 3.3% Latin America and the Caribbean 3.7% 5.4% China 4.5% 6.9% India 3.3% 5.0% Japan 4.9%17.2% Europe8.2%14.7% Italy8.3%18.1% Germany9.7%16.4% Sweden10.3%17.4% U.S.A.8.3%12.3% Source: United Nations 2001.Figure 1. Youthful population.Figure 2. Aged population.Figure 3. Intermediate population.Figure 4. Projected extremely old population.Figure 5. Projected old population.。

morozov偏差原理正则化Morozov's discrepancy principle is a fundamental concept in the field of regularization for ill-posed inverse problems. It provides a criterion for choosing the regularization parameter in the solution of such problems. When solving an ill-posed problem, such as in the case of linear or nonlinear inverse problems, the solution may be sensitive to noise in the data, leading to instability and poor generalization. Regularization techniques are used to mitigate these issues by incorporating additional information or constraints into the problem.The Morozov discrepancy principle addresses the challenge of choosing the regularization parameter, which balances the fidelity of the solution to the data and the stability of the solution. It states that the regularization parameter should be chosen so that the discrepancy between the computed data and the actual data is of the same order as the noise level in the data. In other words, it provides a criterion for stopping theregularization process: the regularization parameter is chosen such that the residual error is approximately equal to the noise level in the data.This principle is important because it provides a practical and data-driven approach to determining the regularization parameter, rather than relying on arbitrary or ad-hoc choices. By aligning the regularization parameter with the noise level in the data, the Morozov discrepancy principle aims to find a balance between fidelity to the data and stability of the solution.In the context of regularization, the choice of the regularization parameter is crucial in controlling the trade-off between the fidelity of the solution and the amount of regularization applied. If the parameter is too small, the solution may overfit the noise in the data, leading to instability and poor generalization. On the other hand, if the parameter is too large, the solution may be overly smooth and fail to capture important features in the data.In summary, the Morozov discrepancy principle provides a principled approach to choosing the regularization parameter in the solution of ill-posed inverse problems. By aligning the regularization parameter with the noise level in the data, it aims to balance the fidelity of thesolution with the stability of the solution, ultimately leading to more robust and reliable solutions.。

References - books1.R. Kress, Linear Integral Equations, Springer-Verlag, New York,1992.2. A. N. Tikhonov, V. Y. Arsenin, On the solution of ill-posed problems, JohnWiley and Sons, New York, 1977.3.H. W. Engl, E. Hanke and A. Neubauer, Regularization of inverse problem,Kluwer, Dordrecht, 1996.4. C. W. Groetsch, The Theory of Tikhonov Regularization for FredholmEquations of the First Kind, Pitman, Boston, 1984.5. C. W. Groetsch, Inverse problem in the Mathematical Sciences, Vieweg,Braunschweig, 1993.6.V. A. Morozov, Regularization Methods for Ill-Posed Problems, CRCPress,1993.7. A. N. Tikhonov, A. S. Leonov and A. G. Yagola, Nonlinear ill-posedproblems, London , New York: Chapman & Hall, 1998.8.O. M. Alifanov, Inverse Heat Transfer Problems, Springer Verlag, 1994.9. A. Kirsch, An Introduction to the Mathematical Theory of Inverse Problem,Springer, 1996.10.C. Susanne, L. Brenner, S. Ridgway, The mathematical Theory of FiniteElement Methods, Springer-Verlag, New York, 1994.11.苏超伟，《偏微分方程逆问题的数值方法及其应用》，12.M. A. Golberg, C. S. Chen, Discrete Projection Methods for IntegralEquations, Computational Mechanics Publications, Southampton, 199713.V. Isakov, Inverse Problems for Partial differential Equations, Springer-Verlag, New York, 1998.14.J. R. Cannon, The one-dimensional heat equation, Addison-Wesley PublishingCompany, 1984.15.R. A. Adams, Sobolev spaces, Pure and Applied Mathematics, Vol. 65.Academic Press, New York-London,1975.16.D. V. Widder, The heat equation, Academic Press, 1975.17.J. V. Beck, K. D. Cole, A. Haji-Sheikh, B. Litouhi, Heat conduction usingGreen’s functions, Hemisphere Publishing Corporation, 1992.18.D. Colton, Inverse acoustic and electromagnetic scattering theory, Springer-Verlag, 1992.19.D. L. Colton, Solution of boundary value problems by the methods of integraloperators, Pitman Publishing, 1976.20.V. Isakov, Inverse Source Problem, AMS Providence, R. I., 1990.21.A. L. Bukhgeim, Introduction to the theory of inverse problem, VSP, 2000.22.G. Wahba, Spline models for observational data, Society for Industrial andApplied Mathematics, 1990.23.C. S. Chen, Y. C. Hon and R. A. Schaback, Scientific Computing with RadialBasis Functions, preprint.24.J.W. Thomas, Numerical partical Differential equations (Finite differencemethods), Springer,1995.25.C. W. Groetsh, Inverse problem activities for undergraduates, 翻译版，程晋，谭永基，刘继军。

novikoff 定理读法英文回答：The Novikoff theorem, also known as the Novikoff convergence theorem, is a fundamental result in machine learning and pattern recognition. It provides conditions under which a sequence of classifiers will converge to the optimal solution. The theorem was first introduced by Alexey Novikoff in 1962.The Novikoff theorem states that if the training data is linearly separable and the learning algorithm satisfies certain conditions, then the sequence of classifiers generated by the algorithm will converge to the optimal solution. In other words, as the algorithm iteratively updates the classifier based on the training data, the classification error will decrease until it reaches zero.To understand the Novikoff theorem, let's consider a simple example. Suppose we have a binary classificationproblem where we want to classify data points into two classes, A and B. We have a training dataset consisting of labeled examples, where each example is represented by a feature vector and a corresponding class label.Now, let's say we initialize our classifier with some initial weights and biases. We then iteratively update the classifier based on the training data using a learning algorithm. As the algorithm progresses, the classifier gets better at classifying the training examples correctly.According to the Novikoff theorem, if the training data is linearly separable, meaning that there exists a hyperplane that can perfectly separate the examples of class A from those of class B, and the learning algorithm satisfies certain conditions, then the sequence of classifiers generated by the algorithm will converge to the optimal solution, where the classification error is zero.In practice, the Novikoff theorem provides theoretical insights into the convergence behavior of learning algorithms. It helps us understand the conditions underwhich a learning algorithm can effectively learn from the training data and converge to the optimal solution. By understanding these conditions, we can design and analyze learning algorithms more effectively.中文回答：Novikoff定理，也被称为Novikoff收敛定理，是机器学习和模式识别中的一个基本结果。

Tikhonov regularizationFrom Wikipedia, the free encyclopediaTikhonov regularization is the most commonly used method of regularization of ill-posed problems named for Andrey Tychonoff. In statistics, the method is also known as ridge regression . It is related to the Levenberg-Marquardt algorithm for non-linear least-squares problems.The standard approach to solve an underdetermined system of linear equations given as,b Ax = is known as linear least squares and seeks to minimize the residual2b Ax -where ∙is the Euclidean norm. However, the matrix A may be ill-conditioned or singular yielding a non-unique solution. In order to give preference to a particular solution with desirable properties, the regularization term is included in this minimization:22x b Ax Γ+-for some suitably chosen Tikhonov matrix , Γ. In many cases, this matrix is chosen as the identity matrix Γ= I , giving preference to solutions with smaller norms. In other cases, highpass operators (e.g., a difference operator or aweighted Fourier operator) may be used to enforce smoothness if the underlying vector is believed to be mostly continuous. This regularization improves the conditioning of the problem, thus enabling a numerical solution. An explicit solution, denoted by , is given by:()b A A A x T T T 1ˆ-ΓΓ+=The effect of regularization may be varied via the scale of matrix Γ. For Γ= αI, when α = 0 this reduces to the unregularized least squares solution provided that (A T A)−1 exists.Contents∙ 1 Bayesian interpretation∙ 2 Generalized Tikhonov regularization∙ 3 Regularization in Hilbert space∙ 4 Relation to singular value decomposition and Wiener filter∙ 5 Determination of the Tikhonov factor∙ 6 Relation to probabilistic formulation∙7 History∙8 ReferencesBayesian interpretationAlthough at first the choice of the solution to this regularized problem may look artificial, and indeed the matrix Γseems rather arbitrary, the process can be justified from a Bayesian point of view. Note that for an ill-posed problem one must necessarily introduce some additional assumptions in order to get a stable solution. Statistically we might assume that a priori we know that x is a random variable with a multivariate normal distribution. For simplicity we take the mean to be zero and assume that each component is independent with standard deviation σx. Our data is also subject to errors, and we take the errors in b to bealso independent with zero mean and standard deviation σb. Under these assumptions the Tikhonov-regularized solution is the most probable solutiongiven the data and the a priori distribution of x, according to Bayes' theorem. The Tikhonov matrix is then Γ= αI for Tikhonov factor α = σb/ σx.If the assumption of normality is replaced by assumptions of homoskedasticity and uncorrelatedness of errors, and still assume zero mean, then theGauss-Markov theorem entails that the solution is minimal unbiased estimate.Generalized Tikhonov regularizationFor general multivariate normal distributions for x and the data error, one can apply a transformation of the variables to reduce to the case above. Equivalently, one can seek an x to minimize22Q P x x b Ax -+- where we have used 2P x to stand for the weighted norm x T Px (cf. theMahalanobis distance). In the Bayesian interpretation P is the inverse covariance matrix of b , x 0 is the expected value of x , and Q is the inverse covariance matrix of x . The Tikhonov matrix is then given as a factorization of the matrix Q = ΓT Γ(e.g. the cholesky factorization), and is considered a whitening filter. This generalized problem can be solved explicitly using the formula()()010Ax b P A Q PA A x T T -++-[edit] Regularization in Hilbert spaceTypically discrete linear ill-conditioned problems result as discretization of integral equations, and one can formulate Tikhonov regularization in the original infinite dimensional context. In the above we can interpret A as a compact operator on Hilbert spaces, and x and b as elements in the domain and range of A . The operator ΓΓ+T A A *is then a self-adjoint bounded invertible operator.Relation to singular value decomposition and Wiener filterWith Γ = αI , this least squares solution can be analyzed in a special way via the singular value decomposition. Given the singular value decomposition of AT V U A ∑=with singular values σi , the Tikhonov regularized solution can be expressed asb VDU x T =ˆwhere D has diagonal values22ασσ+=i iii Dand is zero elsewhere. This demonstrates the effect of the Tikhonov parameter on the condition number of the regularized problem. For the generalized case a similar representation can be derived using a generalized singular value decomposition. Finally, it is related to the Wiener filter:∑==q i i i T i i v b u f x1ˆσ where the Wiener weights are 222ασσ+=i i i f and q is the rank of A . Determination of the Tikhonov factorThe optimal regularization parameter α is usually unknown and often in practical problems is determined by an ad hoc method. A possible approach relies on the Bayesian interpretation described above. Other approaches include the discrepancy principle, cross-validation, L-curve method, restricted maximum likelihood and unbiased predictive risk estimator. Grace Wahba proved that the optimal parameter, in the sense of leave-one-out cross-validation minimizes: ()()[]21222ˆT T X I X X X I Tr y X RSSG -+--==αβτwhereis the residual sum of squares andτ is the effective number degreeof freedom. Using the previous SVD decomposition, we can simplify the above expression: ()()21'22221'∑∑==++-=q i i i i qi i iu b u u b u y RSS ασα ()21'2220∑=++=qi i i i u b u RSS RSS ασαand ∑∑==++-=+-=q i i qi i i q m m 12221222ασαασστ Relation to probabilistic formulationThe probabilistic formulation of an inverse problem introduces (when all uncertainties are Gaussian) a covariance matrix C M representing the a priori uncertainties on the model parameters, and a covariance matrix C D representing the uncertainties on the observed parameters (see, for instance, Tarantola, 2004[1]). In the special case when these two matrices are diagonal and isotropic,and , and, in this case, the equations of inverse theory reduce to the equations above, with α = σD/ σM.HistoryTikhonov regularization has been invented independently in many different contexts. It became widely known from its application to integral equations from the work of A. N. Tikhonov and D. L. Phillips. Some authors use the term Tikhonov-Phillips regularization. The finite dimensional case was expounded by A. E. Hoerl, who took a statistical approach, and by M. Foster, who interpreted this method as a Wiener-Kolmogorov filter. Following Hoerl, it is known in the statistical literature as ridge regression.[edit] References∙Tychonoff, Andrey Nikolayevich (1943). "Об устойчивости обратных задач [On the stability of inverse problems]". Doklady Akademii NaukSSSR39 (5): 195–198.∙Tychonoff, A. N. (1963). "О решении некорректно поставленных задач и методе регуляризации [Solution of incorrectly formulated problemsand the regularization method]". Doklady Akademii Nauk SSSR151:501–504.. Translated in Soviet Mathematics4: 1035–1038.∙Tychonoff, A. N.; V. Y. Arsenin (1977). Solution of Ill-posed Problems.Washington: Winston & Sons. ISBN 0-470-99124-0.∙Hansen, P.C., 1998, Rank-deficient and Discrete ill-posed problems, SIAM ∙Hoerl AE, 1962, Application of ridge analysis to regression problems, Chemical Engineering Progress, 58, 54-59.∙Foster M, 1961, An application of the Wiener-Kolmogorov smoothing theory to matrix inversion, J. SIAM, 9, 387-392∙Phillips DL, 1962, A technique for the numerical solution of certain integral equations of the first kind, J Assoc Comput Mach, 9, 84-97∙Tarantola A, 2004, Inverse Problem Theory (free PDF version), Society for Industrial and Applied Mathematics, ISBN 0-89871-572-5 ∙Wahba, G, 1990, Spline Models for Observational Data, Society for Industrial and Applied Mathematics。

迭代Tikhonov正则化位场向下延拓方法及其在尕林格铁矿的应用赵亚博;刘天佑【摘要】解析延拓是一种广泛应用的位场处理方法，向下延拓可以压制深部地质体的影响，突出浅部异常。

但是，向下延拓滤波因子是一个高通滤波器，会造成下延结果震荡，从而限制了该方法在实际资料中的应用。

文中详细介绍并实现了迭代Tikhonov正则化向下延拓方法，在理论模型上将该方法与传统频率域延拓方法进行对比，表明迭代Tikhonov正则化向下延拓方法的有效性；并将该方法应用于青海尕林格铁矿区磁测资料的处理解释中，下延结果与钻探情况相符，说明在厚覆盖层的勘查区中，运用迭代Tikhonov正则化向下延拓方法能够有效地提高资料处理解释的效果。

%Analytic continuation for potential field is a widely used method for processing and interpretation, because downwardcon⁃tinuation can suppress the influence of deep geological bodies and protrude the shallow layer anomaly. However, the downward con⁃tinuation filter factor is a high⁃pass filter, leading to unstableness of the result, and therefore it can not be used to process the real data. The authors systematically studied and implemented the iterative Tikhonov regularization method for downward continuation of potential fields. In contrast to the continuation of potential field on the theoretical model, the iterative Tikhonov regularization method indicates better effectiveness than frequency domain. The authors also applied this method to Galingeiron deposit's magnetic data pro⁃cessing, and the results indicate that the iteration Tikhonov regularization method for downwardcontinuation of potential fields is wor⁃thy to use in heavy overburden exploration areas.【期刊名称】《物探与化探》【年(卷),期】2015(000)004【总页数】6页(P743-748)【关键词】重磁勘探;向下延拓;迭代Tikhonov正则化;尕林格铁矿【作者】赵亚博;刘天佑【作者单位】中国地质大学武汉地球物理与空间信息学院，湖北武汉 430074;中国地质大学武汉地球物理与空间信息学院，湖北武汉 430074【正文语种】中文【中图分类】P631上世纪70年代初，我国开始将计算机应用于地球物理勘探资料的处理解释。

How I do itOld-fashioned but modern tube cervical esophagostomy Fumiyo Higaki,M.D.*,Masahiro Oishi,M.D.,Takaya Higaki,M.D.,Yuu Hayata,M.D.Department of Surgery,Tottori Municipal Hospital,1-1Matoba,Tottori680-8501,JapanManuscript received June24,2005;revised manuscript November9,2005AbstractBackground:Nasogastric tubes(NGT)are used widely for feeding and decompressions of stomach,but they are associated with several complications and discomfort.When prolonged use of NGT is required,percutaneous endoscopic gastrostomy(PEG)should be considered. However,PEG is not feasible for patients with previous gastrectomy.We have performed tube cervical esophagostomy(TCE)for such difficult cases of PEG.Methods:The current study focused on7patients requiring TCE for feeding or decompression from2004to2005at Tottori Municipal Hospital in Japan.Results:The procedure is relatively simply to perform under local anesthesia and significant complications were not experienced. Conclusions:Feeding or decompressive TCE is useful alternative procedure in patients where PEG is not feasible or unacceptable.©2006 Excerpta Medica Inc.All rights reserved.Keywords:Tube cervical esophagostomy;Percutaneous endoscopic gastrostomy;Feeding tube;Nasogastric tube;Decompression;TerminalNasogastric tubes(NGT)are occasionally used for prolonged periods in hospitalized patients,but in general they are suitable for short-term use because they can cause patients discomfort or complication.When it seems that a patient requires pro-longed use of NGT(Ͼ30days),percutaneous endoscopic gastrostomy(PEG)should be considered.PEG has become established as the optimal method of feeding or decompression since its introduction into clinical practices[1].However,we frequently face technically difficult PEG in patients with previous subtotal gastrectomy,in patients with gastric cancer involving the anterior wall of the stomach, and in patients with interfered organ in puncture route.For such patients in whom PEG is not feasible,we selected tube cervical esophagostomy(TCE).In this report we describe our technique and experience with TCE. PatientsFrom2004to2005,7patients underwent TCE at Tottori Municipal Hospital in Japan.One patient had decompres-sion of the gastrointestinal tract.A58-year-old man with gastric cancer was treated by distal gastrectomy,but the cancer recurred.He showed malignant ileus of carcinoma-tosis and underwent laparotomy twice for palliative bypass. PEG was not feasible due to small residual stomach and bowel adhesion.The remaining6patients underwent the tube feeding.Details of these patients are listed in Table1. TechniqueThe modified technique originally described by Klopp was used in our series[2].All our cases were performed as an isolated procedure under local anesthesia with light se-dation.One percent lidocaine hydrochloride is infiltrated in the subcutaneous tissue,the cervical fascia,and the carotid sheath.The total dose of1%lidocaine hydrochloride is under15mL in most cases.Our surgical technique is as follows.An oblique5-cm incision is made along the anterior border of the left ster-nocleidomastoid muscle.The sternocleidomastoid muscle is retracted laterally and the strap muscles are retracted medi-ally.Opening the carotid sheath,the carotid artery,jugular vein,and vagus nerve are identified and retracted laterally. By general blunt dissection with the indexfinger on the prevertebral muscles and fascia,the dorsal wall of the esophagus can be easily identified.This prevertebral space*Corresponding author.Tel.:ϩ81-857-37-1522;fax:ϩ81-857-37-1553.E-mail address:****************.ne.jpThe American Journal of Surgery192(2006)385–3870002-9610/06/$–see front matter©2006Excerpta Medica Inc.All rights reserved.doi:10.1016/j.amjsurg.2006.06.002is avascular and safe.The dorsal wall of the esophagus is cleared and then pulled out laterally by forceps (Fig.1).Mobilization of the esophagus is only made at the prever-tebral space to avoid injury of the recurrent laryngeal nerve,which is in the esophageal tracheal groove.After mobiliza-tion,the left lateral wall of the esophagus is minimally and carefully denuded to gain space for inserting the tube.The recurrent laryngeal nerve is not intentionally exposed but automatically mobilized anteriorly with the thyroid gland and trachea.The approach route is shown in Fig.2.We make a 0.5-cm longitudinal incision in the dorsal to left lateral wall of the esophagus and insert the 16-F Levin tube through this opening of the esophagus into the stomach under the fluoroscope.A purse string suture is placed around the tube and the skin incision is closed loosely to act as a drain.The Levin tube is brought through the skin incision.The Penrose drain is placed in the dissected mentsThe procedure of TCE was first reported as a case report in 1951by Klopp [2].In 1960,some authors presenteddetail and illustration of this method and reviewed their large series [3,4].Thereafter,TCE was not only performed as isolate procedure but also as a combined procedure with radical neck procedure by head and neck surgeons.In 1980,Gauderer et al first described PEG [1]and this procedure has gained widespread acceptance.Because of the clinical practice of PEG,isolated TCE has been under-utilized and forgotten.In 2004,Mack et al reported the decompressive use of TCE for a patient with gastrointestinal tract obstruction accompanied with linnitis plastica gastric cancer.Attempts at inserting a gastrostomy tube failed in this case [5].In our study,all 7patients were contraindicated for PEG for some reason such as prior gastrectomy,gastric cancer,or bowel adhesion.We performed PEG in 65patients in the same period.Thus,in our hospital,about 10.8%of patients requiring PEG were contraindicated.TCE should be con-sidered in such patients who are not feasible for PEG.TCE may be an old-fashioned and forgotten procedure at the present time,but it is paradoxically a modern techniqueTable 1Details of patients for tube feeding (n ϭ6)Patient Age (y)Sex Cause of dysphagia Reason for contraindication of PEG 189M Cerebral infarction Gastric cancer involving anterior wall 283M Dementia Previous subtotal gastrectomy 378M Dementia Previous subtotal gastrectomy 479M DementiaPrevious subtotal gastrectomy 598M Cerebral infarction Prior peritonitis675MParkinson diseaseInterfered organ in puncture routeM ϭmale;PEG ϭpercutaneous endoscopicgastrostomy.Fig.1.Operation technique.The wall of esophagus is pulled out laterally by forceps.We make a longitudinal incision in the dorsal to lateral wall of the esophagus and insert the Levin tube through this opening of theesophagus.Fig.2.The approach route.By blunt dissection with index finger on the prevertebral muscles and fascia,the dorsal wall of the esophagus can be easily identified.This prevertebral space is avascular and safe.Mobiliza-tion of esophagus is only made at this space to avoid injury of recurrent laryngeal nerve.The black arrow represents the approach route by blunt dissection.386 F.Higaki et al./The American Journal of Surgery 192(2006)385–387in the era of PEG.To know new things,learn by studying the old.The procedure of TCE is relatively simple to perform under local anesthesia and complications associated with procedure are minor.Recent reports revealed no major com-plication in35consecutive patients[6].In our series,there were no major complications such as injury of neck vessels, recurrent laryngeal nerve,and mediastinitis caused by leak-age.Three of our patients had a minor complication,one of subcutaneous emphysema and two of minor leaks from esophagostomy.These leaks resolved with conservative treatment alone and did not cause superior mediastinitis, since the Penrose drain functioned and the Levin tube also acted as a drain.Recently,ultrasonography-guided percutaneous trans-esophageal gastrotubing with a rupture-free balloon was developed[7].We think that the surgical access is safe in two respects.First,in the management of perioperative time we can start adequate drainage by opening the incision of the neck when we suspect superior mediastinitis.Second, the procedure has no risk of inappropriate injury to neck vessels or the thyroid gland.In addition,no special equip-ment was required and an adequate sized tube for feeding or decompression can be used.NGT are esthetically unacceptable for many patients, while esophagostomy tubes are easily hidden underneath clothing and allow freedom from social limitations.To replace the tube is easy after maturation of thefistula tract. In our palliative case,TCE also allowed a liquid diet for oral satisfaction.By use of TCE,we could improve the patient’s quality of life,and palliative care services were successful in the terminal phase.In conclusion,feeding or decompressive TCE is an old-fashioned and forgotten procedure,but is still a useful al-ternative procedure in the era of PEG.References[1]Gauderer MW,Ponsky JL,Izant RJ Jr.Gastrostomy without laparot-omy:a percutaneous endoscopic technique.J Pediatr Surg1980;15: 872–5.[2]Klopp CT.Cervival esophagostomy.J Thorac Cardiovasc Surg1951;21:490–1.[3]Ware L,Garrett WS,Pickrell K.Cervical esophagostomy:a simplifiedtechnic.Ann Surg1967;165:142–4.[4]Ketcham AS,Smith RR.Elective esophagostomy.Am J Surg1962;104:682–5.[5]Mack LA,Pereira J,Temple WJ.Decompressive tube esophagostomy:a forgotten palliative procedure.J Palliat Med2004;7:265–7.[6]Har-el G.Ten-year experience with cervical miniesophagostomy.AnnOtol Rhinol Laryngol1999;108:1111–4.[7]Oishi H,Shindo H,Shirotani N,et al.A nonsurgical technique tocreate an esophagostomy for difficult cases of percutaneous endo-scopic gastrostomy.Surg Endosc2003;17:1224–7.387F.Higaki et al./The American Journal of Surgery192(2006)385–387。

Human Action Recognition Using a Temporal Hierarchyof Covariance Descriptors on3D Joint LocationsMohamed E.Hussein1,Marwan Torki1,Mohammad A.Gowayyed1,Motaz El-Saban2 1Department of Computer and Systems Engineering,Alexandria University,Alexandria,Egypt {mehussein,mtorki,m.gowayyed}@.eg2Microsoft Research Advanced Technology Lab Cairo,Cairo,Egyptmotazel@AbstractHuman action recognition from videos is a chal-lenging machine vision task with multiple im-portant application domains,such as human-robot/machine interaction,interactive entertain-ment,multimedia information retrieval,andsurveillance.In this paper,we present a novel ap-proach to human action recognition from3D skele-ton sequences extracted from depth data.We usethe covariance matrix for skeleton joint locationsover time as a discriminative descriptor for a se-quence.To encode the relationship between jointmovement and time,we deploy multiple covari-ance matrices over sub-sequences in a hierarchicalfashion.The descriptor has afixed length that isindependent from the length of the described se-quence.Our experiments show that using the co-variance descriptor with an off-the-shelf classifica-tion algorithm outperforms the state of the art in ac-tion recognition on multiple datasets,captured ei-ther via a Kinect-type sensor or a sophisticated mo-tion capture system.We also include an evaluationon a novel large dataset using our own annotation.1IntroductionHuman action recognition is one of the many challenging problems targeted by machine vision researchers.It has many important applications in different domains.One of the most active such domains at the moment is interactive entertain-ment.A hive of activity around this domain was recently stimulated by the popularity of several gaming consoles with touch-less interfaces.For truly touch-less interface experi-ence,a gaming console,such as Microsoft’s XBox,deploys a low-cost depth sensor–the Kinect sensor.The depth data captured through the sensor can then be analyzed to estimate the player’s body skeleton in real time[Shotton et al.,2011a], which can further be analyzed to recognize his/her action or gesture.It was conjectured that using skeleton data alone for action recognition can perform better than using other low level image data[Yao et al.,2011].We already know that the approach works quite well in recognizing simple user ges-tures in gaming consoles.Nevertheless,the extent ofsuccess Figure1:Construction of the covariance of3D joints de-scriptor.A sequence of3D joint locations of T=8frames is shown at the top for the“Start System”gesture from the MSRC-12dataset.For the i th frame,the vector of joint coor-dinates,S(i)is formed.The sample covariance matrix is then computed from these vectors.we can achieve with it and its utility in non-entertainment ap-plications are not fully explored yet.In this paper,we address the problem of representing a se-quence of skeletal joint motions over time in a compact and efficient way that is highly discriminative for human action recognition.Particularly,we introduce a novel descriptor for human action recognition that is based on covariance matri-ces.As shown in Figure1,the descriptor is constructed by computing the covariance matrix on the coordinates of body skeleton joints,sampled over time.To encode the tempo-ral dependency of joint locations,we use multiple covariance matrices,each covering a sub-sequence of the input sequence, in a hierarchical fashion.We experimentally evaluated the descriptor on the task of human action recognition.We used multiple(recent and new)datasets of varying sizes and na-tures.In these experiments,classification using our descrip-tor either outperforms the state of the art or is thefirst to be reported.The benefit of the temporal hierarchy of descriptors becomes also evident from our experiments.Proceedings of the Twenty-Third International Joint Conference on Artificial IntelligenceThe paper is organized as follows:The remainder of this section summarizes the related work.In Section2,we give background on the covariance descriptor.In Section3,we explain the proposed Covariance of3D Joints(Cov3DJ)de-scriptor,its different configurations,and efficient computa-tions.Next,in Section4,we present our experimental evalu-ation.Finally,we conclude the paper in Section5.1.1Related WorkIn human action recognition,there are three main challenges to be addressed:data capture,feature descriptors,and action modeling.In this section,we briefly summarize the literature associated with each challenge.Thefirst challenge is the availability and the quality of the captured data.Accurate skeleton data,captured using motion capture systems,such as the CMU MoCap database1,and the HDM05dataset[M¨u ller et al.,2007],are expensive to ac-quire.On the other hand,the Microsoft Kinect,and other low cost depth sensors,make the data acquisition affordable,with a loss of accuracy that is still acceptable for some applica-tions.In addition to the depth maps produced by these sen-sors,the positions of skeletal joints can be estimated[Shotton et al.,2011b].Due to the low cost and widespread of such sensors,several skeletal datasets have been recently released [Li et al.,2010;Fothergill et al.,2012].The second challenge in human action recognition is tofind reliable and discriminative feature descriptions for action se-quences.There are three common types of action descrip-tors:whole sequence,individual frames,and interest points descriptors.The latter two descriptors need additional steps of descriptor aggregation and temporal modeling in order to achieve the recognition goal.An example of the methods thatfind a description of the whole sequence is the moments of Motion History Images [Bobick and Davis,2001;Davis,2001].Examples of other methods thatfind a description for every image in a sequence, and defer the step of learning the dynamics,are the recent works of[Wang et al.,2012b;Xia et al.,2012].In[Wang et al.,2012b],a descriptor of relative positions between pairs of skeletal joints is constructed.The temporal modeling is done in the frequency domain via Fourier Temporal Pyra-mids.In[Xia et al.,2012],a histogram of3D joints descrip-tor in a frame is computed,a dictionary is built and the tem-poral modeling is done via HMM.Examples of methods that use interest point features is the spatio-temporal interest point features STIP[Laptev and Lindeberg,2003].However,local descriptors with depth data lack in its discrimination power due to the lack of texture in depth images.The work pre-sented in[Gowayyed et al.,2013],which uses histograms of displacement orientations,is the closest in spirit to the work presented in this paper.The third challenge is modeling the dynamics of an ac-tion.Sequence analysis via generative models,such as HMMs[Xia et al.,2012],or discriminative models,such as CRFs[Han et al.,2010],are usually employed.In such methods,the joint positions or histograms of the joints po-sitions are used as observations.Other recent approaches 1/use recurrent neural networks[Martens and Sutskever,2011],or Conditional Restricted Boltzman Machines[Mnih et al.,2011].Due to the large number of parameters to be estimated,these models need large amounts of data samples and trainingepochs to accurately estimate its model parameters.2The Covariance DescriptorThe covariance matrix for a set of N random variables is anN×N matrix whose elements are the covariance between every pair of variables.Let X be a vector of N random vari-ables.The covariance matrix of the random vector X is de-fined as C OV(X)=E[(X−E(X))(X−E(X)) ],where E()isthe expectation operator.The covariance matrix encodes in-formation about the shape of the joint probability distributionof the set of random variables.The covariance matrix wasfirst introduced as a descrip-tor by Tuzel,et al.[2006].In this work,the descriptor wasused to describe a region of an image,where variables cor-responded to different feature maps computed for the region,and samples of each variable corresponded to values of itsfeature map at different pixels in the region.The descriptorwas applied successfully on object detection,texture classifi-cation,and pedestrian detection[Tuzel et al.,2008].Recently,the same idea was generalized to video se-quences by considering features of pixels in a volumetricspatio-temporal patch,and was applied to action recognition [Sanin et al.,2013].In this paper,we take a different ap-proach inspired by thefindings of[Yao et al.,2011],in whichpose data was found to outperform low-level appearance fea-tures in action recognition.Particularly,we use the pose data,represented by the body joint locations,sampled over time,as the variables on which the covariance matrix is computed. 3The Covariance of3D Joints(Cov3DJ) DescriptorSuppose that the body is represented by K joints,and the action is performed over T frames.Let x(t)i,y(t)i,and z(t)i be the x,y,and z coordinates of the i th joint at frame t.Let S be the vector of all joint locations,that is S= [x1,...,x K,y1,...,y K,z1,...,z K] ,which has N=3K el-ements.Then,the covariance descriptor for the sequence is C OV(S).Typically,the probability distribution of S is not known and we use the sample covariance instead,which is given by the equationC(S)=1T−1Tt=1(S−¯S)(S−¯S) ,(1)where¯S is the sample mean of S,and the is the transpose operator.The sample covariance matrix2,C(S),is a symmetric N×N matrix.For the descriptor,we only use its upper triangle. For example,for a skeleton with20joints,such as the one produced by the Kinect sensor(examples are in Figure1), N=3×20=60.The upper triangle of the covariance matrix in this case is N(N+1)/2=1830,which is the length of the descriptor.2Referred to just as’covariance matrix’for the rest of the paper.Figure2:Temporal construction of the covariance descriptor.C li is the i th covariance matrix in the l th level of the hierar-chy.A covariance matrix at the l th level covers T2l frames ofthe sequence,where T is the length of the entire sequence.3.1Temporal Hierarchical ConstructionThe Cov3DJ descriptor captures the dependence of locations of different joints on one another during the performance of an action.However,it does not capture the order of motion in time.Therefore,if the frames of a given sequence are ran-domly shuffled,the covariance matrix will not change.This could be problematic,for example,when two activities are the reverse temporal order of one another,e.g.“push”and “pull”.To add the temporal information to Cov3DJ,we use a hi-erarchy of Cov3DJs,which is inspired by the idea of spatial pyramid matching[Lazebnik et al.,2006]in2D images.The hierarchical construction is shown in Figure2.The top level Cov3DJ is computed over the entire video sequence.The lower levels are computed over smaller windows,overlap-ping or non-overlapping,of the entire sequence.Figure2 shows only two levels in the hierarchy.Each covariance ma-trix is identified by two indices:thefirst is the hierarchy level index,and the second is the index within the level.The top level matrix covers the entire sequence and is denoted by C00.A covariance matrix at level l is computed over T/2l frames of the sequence.The step from one window to the next is ei-ther the length of the window or half of it.If the step is half the window length,the windows overlap with one another.In Figure2,covariance matrices in the second level overlap. As we show in Section4adding more levels and allow-ing overlap enhances the ability of a classifier to distinguish among actions using the descriptor.However,the more lay-ers we add and allowing overlap increases the length of the descriptor.For the descriptor configurations in Figure2,a skeleton represented with20joints results in a descriptor of length4×1830=7320.Fast Descriptor ConstructionCreating multiple layers of the temporal hierarchy and allow-ing overlap dictates computing multiple covariance matrices for sub-sequences of the same sequence.Luckily,a dynamic programming approach can be deployed to make the the com-putation of every element of the matrix possible in constant time,after some pre-computations are performed.A simi-lar idea was used in prior work with the names integral im-ages for covariances on image patches[Tuzel et al.,2008], and integral videos for covariances on spatio-temporal video patches[Sanin et al.,2013].The same concept can be applied in our case with the distinction that integrals are needed only on the time dimension,which we refer to as integral signals. Following similar notation to[Tuzel et al.,2008],we de-fine the two integral signals P(t)and Q(t)asP(t)=ti=1S(i),Q(t)=ti=1S(i)S(i) .(2)After some algebraic manipulation,we can reach the follow-ing formula for computing the covariance matrix of the range of frames from t1+1to t2,inclusively.C(t1,t2)(S)=1M−1(Q(t1,t2)−1MP(t1,t2)P(t1,t2) ),(3)where M=t2−t1,Q(t1,t2)=Q(t2)−Q(t1),and P(t1,t2)= P(t2)−P(t1).Details of the derivation are a straight forward simplification of the corresponding2D version in[Tuzel et al.,2008].Having computed the signal integrals,P and Q,we can compute the covariance matrix over any range of frames in time that is independent of the length of the range,using Equation3.It is worth noting here that integrating over one dimen-sion only in integral signals,compared to integration over two and three dimensions in integral images and integral videos, respectively,is not just a simplification of mathematics and computational demands.It also leads to significantly less er-ror accumulation on computing the integrals[Hussein et al., 2008].4ExperimentsWe evaluated the discrimination power of our descriptor for action recognition.We performed this evaluation on three publicly available datasets.In one of them,we used our own annotation.Two of the datasets were acquired using a Kinect sensor,and one using a motion capture system.Details of the experiments are presented in the following sub-sections.In all experiments,we used a linear SVM classifier,using the LIBSVM software[Chang and Lin,2011]with the descrip-tor.Before training or testing,descriptors are normalized to have unit L2norms.The covariance matrix is shift invariant by nature.To make it scale invariant,we normalized joint coordinates over the sequence to range from0to1in all di-mensions before computing the descriptor.4.1MSR-Action3D DatasetThe MSR-Action3D dataset[Li et al.,2010]has20action classes performed by10subjects.Each subject performed each action2or3times.There are567sequences in total, from which we used5443,each was recorded as a sequence of depth maps and a sequence of skeletal joint locations.Both types of sequences were acquired using a Kinect sensor.20 joints were marked in the skeletal joint sequences as shown in Figure3.3[Li et al.,2010]already excluded10sequences out of the567 in their experiments.We excluded13more sequences that we found severely corrupted.Figure3:Skeleton joint locations and names as captured bythe Kinect sensor.Method Acc.(%) Rec.Neural Net.[Martens and Sutskever,2011]42.50 Hidden Markov Model[Xia et al.,2012]78.97 Action Graph[Li et al.,2010]74.70 Random Occupancy Patterns[Wang et al.,2012a]86.50 Actionlets Ensemble[Wang et al.,2012b]88.20 Proposed Cov3DJ90.53 Table1:Comparative results on the MSR-Action3D dataset. We used the typical experimental setup on this dataset[Liet al.,2010],which divides the action classes into three action sets,each containing8action classes,with some overlap be-tween action sets.Classifiers are trained to distinguish amongactions in the same set only.The reported accuracy is the av-erage over the three sets.Several studies have already been conducted on the MSR-Action3D dataset.Table1shows the classification rate ofour approach compared to the state-of-the-art methods4.Ourresults in this table correspond to using three levels of thedescriptor while allowing overlap in the second and third lev-els.Our approach achieves90.53%classification rate,ex-ceeding the second best approach by more than2%.It isworth noting that we only rely on joint locations in our ap-proach,while other algorithms,such as[Li et al.,2010;Wang et al.,2012a],use the depth maps.Moreover,ourdescriptor construction and classification algorithm are con-siderably simpler than the Ensemble of Actionlets used in [Wang et al.,2012b];and,our encoding of temporal informa-tion is also considerably simpler than HMMs,used in[Xiaet al.,2012].Therefore,the effectiveness of our approach is fostered by its simplicity compared to other state of the artmethods,which shows its practical advantage.Next,we use the same dataset to evaluate the effect of 4The entry for RNN in Table1was copied from[Wang et al., 2012b].L=1L=2L=3L=2,OL L=3,OL AS188.0486.9686.9688.0488.04AS278.5781.2584.8283.9389.29AS395.2494.2993.3394.2994.29 Mean87.2887.5088.3788.7590.53Table2:Results on MSR-Action3D using different levels in the temporal hierarchy.Adding levels and allowing overlap (marked with OL)enhances classification accuracy in gen-eral.Best results for L=3with overlapping.Metaphoric No.of Iconic No.ofGestures Insts.Gestures Insts.Start system508Duck500Push right522Goggles508Wind it up649Shoot511Bow507Throw515 Had enough508Change weapon498Beat both516Kick502Table3:Gesture classes in the MSRC-12dataset and the number of annotated instances from each class.changing the parameters of descriptor construction.The re-sults in Table2show the classification accuracy for different levels in the temporal hierarchy while enabling or disabling overlap.In general,we can deduce that adding more levels enhances the descriptor’s discrimination power,and hence, the classification accuracy.The overlap also enhances the classification accuracy.Another observation is that even with one level,Cov3DJ outperforms all algorithms in Table1,ex-cept for the Actionlets Ensemble[Wang et al.,2012b].With only two levels and overlap,Cov3DJ outperforms all other methods in the table.4.2MSRC-12Kinect Gesture DatasetTo test the proposed approach when a large number of train-ing instances is available,we experimented on the MSRC-12dataset[Fothergill et al.,2012].MSRC-12is a relatively large dataset for action/gesture recognition from3D skeleton data,recorded using a Kinect sensor.The dataset has594se-quences,containing the performances of12gestures by30 subjects.There are6,244annotated gesture instances in to-tal.Table3shows the12gesture classes in the dataset and the number of annotated instances from each.The gesture classes are divided into two groups:metaphoric gestures,and iconic gestures.Each sequence in the dataset is a recording of one subject performing one gesture for several times in a row.The ground truth annotation for each sequence marks the action point of the gesture,which is defined in[Nowozin and Shotton,2012] as”a single time instance at which the presence of the action is clear and that can be uniquely determined for all instances of the action”.For a real-time application,such as a game, this is the point at which a recognition module is required to detect the presence of the gesture.The ground truth annotation of the dataset is designed for experimenting on the task of action detection,in which it is required to locate instances of different actions in a givenvideo sequence.We wanted to benefit from the large volume of the dataset without moving away from the task of action recognition.Therefore,we needed to know the start and end of each gesture instance,not just the action point.To perform our action recognition experiments,we manu-ally annotated the sequences of the dataset to mark the onset and offset of each gesture instance.5To make this task easy, we made use of the action point annotation.We developed a simple tool to facilitate locating the boundaries of each action instance starting the search always from the marked action point.The lengths of the gesture instances–i.e.the number of frames between the onset and offset of the gesture–result-ing from our annotation range from13frames to492frames. The lower end of this range correspond to legitimate instances of the gesture“wind it up”,which is sometimes performed by the subject multiple times in a row,and the ground truth marks each as a separate instance.The higher end of the range,however,typically correspond to an odd performance of the gesture,e.g.dancing and moving back and forth while performing the gesture with unnecessary repetitions,or an ex-tra slow performance of the gesture.Such odd long instances constitute a very small fraction of the dataset.Only40in-stances in the entire dataset are longer than200frames.We included all instances in our experiments.The median length of an instance is80frames.If we consider the ending of the gesture instance to be the action point,instead of the offset point,the median length becomes40frames and the maxi-mum becomes440frames.Given the wide range of gesture lengths,choosing afixed length sequence ending at an action point,e.g.35frames as in[Fothergill et al.,2012],to rep-resent an action is not quite convincing.While35frames is shorter than more than half of the instances,it may include more than two consecutive instances of the short gestures, such as“wind it up”.In the following subsections,wefirst present the experi-mental results using our own annotation,which are thefirst such results on this dataset.Then,we compare to a recently published experiment on the same dataset in an action recog-nition setting[Ellis et al.,2013],in which gesture boundaries where considered to be the mid-points between consecutive action points.In all these experiments,we vary the number of levels in the temporal hierarchy,between1and2,and en-able or disable overlap in the latter.Given the large volume of the dataset,we could not experiment on three hierarchical levels of the descriptor due to the limitations of our system’s configurations.We completely disregard the type of instruc-tions given to each subject[Fothergill et al.,2012]in all our experiments.Leave-One-Out ExperimentsIn this experiment,we used all action instances from29sub-jects for training and the action instances of the remaining subject for testing.We performed the experiment30times, excluding one subject in each run.The benefit of such setup is two fold:First it allows for testing the inter-subject gener-alization of the approach while using as much data as possible 5This annotation can be downloaded from .eg/mehussein/msrc12annot4rec/.L=1L=2L=2,OL Leave One Out92.793.693.650%Subject Split90.391.291.71/3Training97.797.897.92/3Training98.698.798.7[Ellis et al.,2013]’s89.690.991.2Table4:Classification accuracy results for experiments on the MSRC-12dataset with different experimental setups and different descriptor configurations.The numbers shown are percentages.For explanation of each experimental setup,re-fer to subsections of Section4.2.for training.Second,it allows for detecting problematic sub-jects and analyzing the sources of some of the classification errors.The results of the experiment are shown in thefirst row of Table4.The values in the table are the average classification rate over the30runs.This average value ranges from92.7% to93.6%,slightly increasing with increasing the descriptor’s length(by adding an extra level to the hierarchy and allowing overlap).The high classification rate verifies the inter-subject discrimination power of the descriptor.Inspection of the individual errors in each of the30runs revealed that the most problematic gesture instances belonged to subject number2.By inspecting the gesture classes with high error rate for this subject,we found that in most cases, the subject performed the gesture with unrelated movements, for example,dancing or walking while the gesture should be performed only by hands.50%Subject SplitIn the next experiment,we test the sensitivity of the classifier to reducing the number of training samples.We trained20 different classifiers,each on a random selection of half the persons for training and the other half for testing.The av-erage correct classification rate,as shown in the second row of Table4,ranges from90.3%to91.7%.Despite using only half the instances for training compared to around97%in the leave-one-out experiment,the reduction in the classification accuracy is less than3%From the experiments above,it is clear that the discrim-ination power of our descriptor is larger on the MSRC-12 dataset.This is despite the larger number of gestures and de-spite the larger differences among subjects’performances of the same gesture.This can be attributed to the larger number of instances available to train the classifiers.1:2Instance SplitIn thefinal experiment,we test how much the classifier’s per-formance can be enhanced if samples from all subjects are used in training and testing.The instances of each gesture class are randomly split between training and testing.The splits are done by random sampling without replacement.In other words,no instance can be shared between the training and testing sets.Two different split ratios are used:either 1/3of the instances are used for training and the rest for test-ing,or2/3are used for training and the rest for testing.20 different random splits from each ratio are generated in each ratio.The results for this experiment are shown in the third and forth rows of Table4.When one third of the data is used for training,the accuracy is around98%.When two thirds are used for training,the accuracy goes up to around99%. From this experiment,we can see that a significant portion of the error we saw in the previous experiments were due to inter-person variations in performing the gesture.This could be due to giving different types of instructions to different users upon collecting the dataset[Fothergill et al.,2012]. Using Mid-Action Points as Gesture BoundariesIn this section,we compare to the results of[Ellis et al., 2013],in which a4-fold cross validation experiment was conducted on the MSRC-12dataset.Following their same setup,the midpoint between two consecutive action points were used to divide a video into gesture instances,while us-ing thefirst and last frames of a video as the boundaries for thefirst and last gesture instances.The results of this experi-ment are shown in the last row of Table4.The classification rate reported in[Ellis et al.,2013]is88.7%,which is slightly inferior to our basic configuration.Our best configuration achieves91.2%accuracy in this experiment.4.3HDM05-MoCap DatasetSimilar to[Ofli et al.,2012],we experimented with our approach on a Motion Capture dataset,namely the HDM05 database[M¨u ller et al.,2007].There are three main dif-ferences between this dataset and the preceding two datasets: First,it is captured using motion-capture sensors,which leads to much less noise than in the data acquired by a Kinect sen-sor.Second,the number of joints recorded is31instead of 20.This leads to a longer descriptor since the size of the covariance matrix in this case is93×93.Third,the frame rate is much higher,120fps instead of15or30fps as in the preceding two datasets.We used the same setup in[Ofli et al.,2012]with the same 11actions performed by5subjects.We had249sequences in total.We used3subjects(140action instances)for train-ing,and2subjects(109action instances)for testing.The set of actions used in this experiment is:depositfloor,elbow to knee,grab high,hop both legs,jog,kick forward,lie down floor,rotate both arms backward,sneak,squat,and throw bas-ketballThe results in Table5show that the most basic configura-tion of our descriptor outperforms the best configuration of the SMIJ approach[Ofli et al.,2012].The results also show that the more levels we add to the temporal hierarchy the bet-ter classification accuracy we can achieve.We can observe how the classification accuracy with the HDM05dataset is significantly better than the classification accuracy with the MSR-Action3D dataset(Section4.1)al-though the numbers of training samples used in both datasets are comparable.This can be attributed to the much lower level of noise in HDM05’s data,and the extra information available with the higher frame rate and larger number of joints.Method Accuracy(%)SMIJ[Ofli et al.,2012]84.40Cov3DJ L=192.66Cov3DJ L=293.57Cov3DJ L=395.41Table5:Classification accuracy on the HDM05dataset with various configurations of Cov3DJ compared to the baseline method.5Conclusion and Future DirectionsWe introduced a novel descriptor for action sequences con-sisting of3D skeletal joint movements.The descriptor, named Cov3DJ,is based on the covariance matrix of3D joint locations over the entire sequence.Cov3DJ has afixed length,independent from the sequence’s length.Temporal information can be effectively encoded in the descriptor by incorporating multiple Cov3DJs for sub-windows of the se-quence,that are possibly overlapping,in a temporal hierar-chy.Cov3DJ is also efficient to compute over multiple over-lapping windows using integral signals.We evaluated the discrimination power of the descriptor on the task of human action recognition from skeleton data. Despite the simplicity of the descriptor,training an off-the-shelf linear SVM classifier on it outperforms the state of the art methods in multiple dataset.We achieved a classification rate of90.5%on the MSR-Action3D dataset,and95.4%on HDM05MoCap dataset.In addition,we experimented on the newly introduced MSRC-12dataset,with our own annota-tion,achieving up to93.6%cross-subject classification rate. While the descriptor is both scale and translation invari-ant,it is not rotation or reflection invariant.We believe that rotation invariance can easily be achieved by transforming joint locations to the subject’s local coordinate frame,defined based on the current pose,instead of using the camera frame. Alternatively,rotation invariant features,such as angles or ve-locities,can be used instead of joint locations.Reflection in-variance,on the other hand,is more challenging to achieve. However,in some applications,actions performed by the left and right parts of the body are distinct,in which case,reflec-tion invariance is not desired.Finally,a moreflexible tempo-ral sub-division may help achieve better performance. AcknowledgmentThis research project was sponsored by a grant from the Mi-crosoft Research Advanced Technology Lab Cairo,and was hosted by VT-MENA Research Center in Alexandria Univer-sity.References[Bobick and Davis,2001]A.F.Bobick and J.W.Davis.The recognition of human movement using temporal tem-plates.Pattern Analysis and Machine Intelligence,IEEE Transactions on,23(3):257–267,mar2001.[Chang and Lin,2011]Chih-Chung Chang and Chih-Jen Lin.LIBSVM:A library for support vector machines.。

REFERENCESAbramowitz,M.and Stegun,I.A.(Editors),Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables,National Bureau of Standards Applied Mathematics, Washington,1964.Acrivos,A.,A note of the rate of heat or mass transfer from a small sphere freely suspended in linear shearfield,J.Fluid Mech.,V ol.98,No.2,pp.299–304,1980.Aksenov,A.V.,Linear differential relations between solutions of the equations of Euler–Poisson–Darboux class,Mechanics of Solids,V ol.36,No.1,pp.11–15,2001.Akulenko,L.D.and Nesterov,S.V.,Determination of the frequencies and forms of oscillations of non-uniform distributed systems with boundary conditions of the third kind,Appl.Math.Mech.(PMM),V ol.61,No.4,p.531–538,1997.Akulenko,L.D.and Nesterov,S.V.,Vibration of an nonhomogeneous membrane,Mechanics of Solids,V ol.34,No.6,pp.112–121,1999.Akulenko,L.D.and Nesterov,S.V.,Free vibrations of a homogeneous elliptic membrane, Mechanics of Solids,V ol.35,No.1,pp.153–162,2000.Akulenko,L.D.,Nesterov,S.V.,and Popov,A.L.,Natural frequencies of an elliptic plate with clamped edge,Mechanics of Solids,V ol.36,No.1,pp.143–148,2001.Andreev,V.K.,Kaptsov,O.V.,Pukhnachov,V.V.,and Rodionov,A.A.,Applications of Group-Theoretical Methods in Hydrodynamics,Nauka,Moscow,1994.(English translation:Kluwer, Dordrecht,1999.)Appell,P.,Trait´e de M´e canique Rationnelle,T.1:Statique.Dinamyque du Point(Ed.6),Gauthier-Villars,Paris,1953.Arscott,F.,Periodic Differential Equations,Macmillan(Pergamon),New York,1964. Arscott,F.,The Whittaker–Hill equation and the wave equation in paraboloidal coordinates,Proc.Roy.Soc.Edinburg,V ol.A67,pp.265–276,1967.Babich,V.M.,Kapilevich,M.B.,Mikhlin,S.G.,et al.,Linear Equations of Mathematical Physics [in Russian],Nauka,Moscow,1964.Batchelor,G.K.,Mass transfer from a particle suspended influid with a steady linear ambient velocity distribution,J.Fluid Mech.,V ol.95,No.2,pp.369–400,1979.Bateman,H.and Erd´e lyi,A.,Higher Transcendental Functions,Vol.1and Vol.2,McGraw-Hill, New York,1953.Bateman,H.and Erd´e lyi,A.,Higher Transcendental Functions,Vol.3,McGraw-Hill,New York, 1955.Bateman,H.and Erd´e lyi,A.,Tables of Integral Transforms,Vol.1and Vol.2,McGraw-Hill,New York,1954.Belotserkovskii,O.M.,and Oparin,A.A.,Numerical Experiment in Turbulence,Nauka,Moscow, 2000.Beyer,W.H.,CRC Standard Mathematical Tables and Formulae,CRC Press,Boca Raton,1991. Bitsadze,A.V.and Kalinichenko,D.F.,Collection of Problems on Mathematical Physics Equations [in Russian],Nauka,Moscow,1985.Bolotin V.V.(Editor),Vibration in Engineering:a Handbook.Vol.1.Vibration of Linear Systems [in Russian],Mashinostroenie,Moscow,1978.Borzykh,A.A.and Cherepanov,G.P.,A plane problem of the theory of convective heat transfer and mass exchange,PMM[Applied Mathematics and Mechanics],V ol.42,No.5,pp.848–855, 1978.Boyer,C.,The maximal kinematical invariance group for an arbitrary potential,Helv.Phys.Acta, V ol.47,pp.589–605,1974.Boyer,C.,Lie theory and separation of variables for equation+2−(21+22)=0, SIAM J.Math.Anal.,V ol.7,pp.230–263,1976.Bˆo cher,M.,Die Reihenentwickelungen der Potentialtheory,Leipzig,1894.Brenner,H.,Forced convection-heat and mass transfer at small Peclet numbers from particle of arbitrary shape,Chem.Eng.Sci.,V ol.18,No.2,pp.109–122,1963.Brychkov,Yu.A.and Prudnikov,A.P.,Integral Transforms of Generalized Functions,Gordon& Breach Sci.Publ.,New York,1989.Budak,B.M.,Samarskii,A.A.,and Tikhonov,A.N.,Collection of Problems on Mathematical Physics[in Russian],Nauka,Moscow,1980.Burde,G.I.,The construction of special explicit solutions of the boundary-layer equations.Steady flows,Q.J.Mech.Appl.Math.,V ol.47,No.2,pp.247–260,1994.Butkov,E.,Mathematical Physics,Addison-Wesley,Reading,Mass.,1968.Butkovskiy,A.G.,Characteristics of Systems with Distributed Parameters[in Russian],Nauka, Moscow,1979.Butkovskiy,A.G.,Green’s Functions and Transfer Functions Handbook,Halstead Press–John Wiley&Sons,New York,1982.Carslaw,H.S.and Jaeger,J.C.,Conduction of Heat in Solids,Clarendon Press,Oxford,1984. Clarkson,P.A and Kruskal,M.D.,New similarity reductions of the Boussinesq equation,J.Math.Phys.,V ol.30,No.10,pp.2201–2213,1989.Colton,D.,Partial Differential Equations.An Introduction,Random House,New York,1988. Courant,R.and Hilbert,D.,Methods of Mathematical Physics,Vol.2,Wiley–Interscience Publ., New York,1989.Crank,J.,The Mathematics of Diffusion,Clarendon Press,Oxford,1975.Davis,B.,Integral Transforms and Their Applications,Springer-Verlag,New York,1978. Davis,E.J.,Exact solutions for a class of heat and mass transfer problems,Can.J.Chem.Eng., V ol.51,No.5,pp.562–572,1973.Deavours,C.A.,An exact solution for the temperature distribution in parallel plate Poiseuilleflow, Trans.ASME,J.Heat Transfer,V ol.96,No.4,1974.Dezin,A.A.,Partial Differential Equations.An Introduction to a General Theory of Linear Boundary Value Problems,Springer-Verlag,Berlin-New York,1987.Ditkin,V.A.and Prudnikov,A.P.,Integral Transforms and Operational Calculus,Pergamon Press,New York,1965.Doyle,Ph.W.,Separation of variables for scalar evolution equations in one space dimension,J.Phys.A:Math.Gen.,V ol.29,pp.7581–7595,1996.Doyle,Ph.W.and Vassiliou,P.J.,Separation of variables for the1-dimensional non-linear diffusion equation,Int.J.Non-Linear Mech.,V ol.33,No.2,pp.315–326,1998.Elrick,D.E.,Source functions for diffusion in uniform shearflows,Australian J.Phys.,V ol.15, No.3,p.283–288,1962.Faddeev,L.D.(Editor),Mathematical Physics.Encyclopedia[in Russian],Bol’shaya Rossiiskaya Entsyklopediya,Moscow,1998.Faminskii,A.V.,On mixed problems for the Corteveg–de Vries equation with irregular boundary data,Doklady Mathematics,V ol.59,No.3,pp.366–367,1999.Farlow,S.J.,Partial Differential Equations for Scientists and Engineers,John Wiley&Sons,New York,1982.Galaktionov,V.A.,Invariant subspace and new explicit solutions to evolution equations with quadratic nonlinearities,Proc.Roy.Soc.Edinburgh,V ol.125A,No.2,pp.225–448,1995. Galaktionov,V.A.and Posashkov,S.A.,On new exact solutions of parabolic equations with quadratic nonlinearities,Zh.Vych.Matem.i Mat.Fiziki,V ol.29,No.4,pp.497–506,1989. Galaktionov,V.A.and Posashkov,S.A.,Exact solutions and invariant subspace for nonlinear gradient-diffusion equations,Zh.Vych.Matem.i Mat.Fiziki,V ol.34,No.3,pp.374–383,1994. Galaktionov,V.A.,Posashkov,S.A.,and Svirshchevskii,S.R.,Generalized separation of variables for differential equations with polynomial right-hand sides,Dif.Uravneniya,V ol.31, No.2,pp.253–261,1995.Gel’fand,I.M.and Shilov,G.E.,Distributions and Operations on Them[in Russian],Fizmatlit, Moscow,1959.Gradshteyn,I.S.and Ryzhik,I.M.,Tables of Integrals,Series,and Products,Academic Press, Orlando,2000.Graetz,L.,¨Uber die Warmeleitungsf¨a higkeit von Fl¨u ssigkeiten,Annln.Phys.,Bd.18,S.79–84, 1883.Grundland,A.M.and Infeld,E.,A family of nonlinear Klein–Gordon equations and their solutions, J.Math.Phys.,V ol.33,No.7,pp.2498–2503,1992.Guenther,R.B.and Lee,J.W.,Partial Differential Equations of Mathematical Physics and Integral Equations,Dover Publ.,Mineola,1996.Gupalo,Yu.P.,Polyanin,A.D.,and Ryazantsev,Yu.S.,Mass Exchange of Reacting Particles with Flow[in Russian],Nauka,Moscow,1985.Gupalo,Yu.P.and Ryazantsev,Yu.S.,Mass and heat transfer from a sphere in a laminarflow, Chem.Eng.Sci.,V ol.27,pp.61–68,1972.Haberman,R.,Elementary Applied Partial Differential Equations with Fourier Series and Boundary Value Problems,Prentice-Hall,Englewood Cliffs,1987.Hanna,J.R.and Rowland,J.H.Fourier Series,Transforms,and Boundary Value Problems, Wiley-Interscience Publ.,New York,1990.Happel,J.and Brenner,H.,Low Reynolds Number Hydrodynamics,Prentice-Hall,Englewood Cliffs,1965.H¨o rmander,L.,The Analysis of Linear Partial Differential Operators.II.Differential Operators with Constant Coefficients,Springer-Verlag,Berlin-New York,1983.H¨o rmander,L.,The Analysis of Linear Partial Differential Operators.I.Distribution Theory and Fourier Analysis,Springer-Verlag,Berlin,1990.Ibragimov N.H.(Editor),CRC Handbook of Lie Group to Differential Equations,Vol.1,CRC Press,Boca Raton,1994.Ignatovich,N.V.,Invariant irreducible,partially invariant solutions of stationary boundary layer equations,Mat.Zametki,V ol.53,No.1,pp.140–143,1993.Ivanov,V.I.,and Trubetskov,M.K.,Handbook of Conformal Mapping with Computer-Aided Visualization,Boca Raton,CRC Press,1994.John,F.,Partial Differential Equations,Springer-Verlag,New York,1982.Kalnins,E.,On the separation of variables for the Laplace equation in two-and three-dimensional Minkowski space,SIAM J.Math.Anal.,Hung.,V ol.6,pp.340–373,1975.Kalnins,E.and Miller,W.(Jr.),Lie theory and separation of variables,5:The equations +=0and+−Kalnins,E.and Miller,W.(Jr.),Lie theory and separation of variables,8:Semisubgroup coordinates for−2=0,J.Math.Phys.,V ol.16,pp.2507–2516,1975.Kalnins,E.and Miller,W.(Jr.),Lie theory and separation of variables,9:Orthogonal-separable coordinate systems for the wave equation−2=0,J.Math.Phys.,V ol.17,pp.331–335, 1976.Kalnins,E.and Miller,W.(Jr.),Lie theory and separation of variables,10:Nonorthogonal-separable solutions of the wave equation−2=0,J.Math.Phys.,V ol.17,pp.356–368, 1976.Kamke,E.,Differentialgleichungen:L¨o sungsmethoden und L¨o sungen,I,Gew¨o hnliche Differential-gleichungen,B.G.Teubner,Leipzig,1977.Kamke,E.,Differentialgleichungen:L¨o sungsmethoden und L¨o sungen,II,Partielle Differential-gleichungen Erster Ordnung f¨u r eine gesuchte Funktion,Akad.Verlagsgesellschaft Geest& Portig,Leipzig,1965.Kanwal,R.P.,Generalized Functions.Theory and Technique,Academic Press,Orlando,1983. Korn,G.A.and Korn,T.M.,Mathematical Handbook for Scientists and Engineers,McGraw-Hill, New York,1968.Koshlyakov,N.S.,Gliner,E.B.,and Smirnov,M.M.,Partial Differential Equations of Mathematical Physics[in Russian],Vysshaya Shkola,Moscow,1970.Krein,S.G.(Editor),Functional Analysis[in Russian],Nauka,Moscow,1972.Krylov,A.N.,Collected Works:III Mathematics,Pt.2[in Russian],Izd-vo AN SSSR,Moscow, 1949.Lamb,H.,Hydrodynamics,Dover Publ.,New York,1945.Lavrent’ev,M.A.and Shabat B.V.,Methods of Complex Variable Theory[in Russian],Nauka, Moscow,1973.Lavrik,V.I.and Savenkov,V.N.,Handbook of Conformal Mappings[in Russian],Naukova Dumka,Kiev,1970.Landau,L.D.and Lifshits,E.M.,Quantum Mechanics.Nonrelativistic Theory[in Russian], Nauka,Moscow,1974.Lebedev,N.N.,Skal’skaya,I.P.,and Uflyand,Ya.S.,Collection of Problems on Mathematical Physics[in Russian],Gostekhizdat,Moscow,1955.Leis,R.,Initial-Boundary Value Problems in Mathematical Physics,John Wiley&Sons,Chichester, 1986.Levich,V.G.,Physicochemical Hydrodynamics,Prentice-Hall,Englewood Cliffs,New Jersey,1962. Levitan,B.M.and Sargsyan,I.S.,Sturm–Liouville and Dirac Operators[in Russian],Nauka, Moscow,1988.Loitsyanskiy,L.G.,Mechanics of Liquids and Gases,Begell House,New York,1996. Lykov,A.V.,Theory of Heat Conduction[in Russian],Vysshaya Shkola,Moscow,1967. Mackie,A.G.,Boundary Value Problems,Scottish Academic Press,Edinburgh,1989. Makarov,A.,Smorodinsky,J.,Valiev,K.,and Winternitz,P.,A systematic search for nonrelativistic systems with dynamical symmetries.Part I:The integrals of motion,Nuovo Cimento,V ol.52A,pp.1061–1084,1967.Marchenko,V.A.,Sturm–Liouville Operators and Applications,Birkhauser Verlag,Basel-Boston, 1986.Markeev,A.P.,Theoretical Mechanics[in Russian],Nauka,Moscow,1990.Mathematical Encyclopedia[in Russian],Sovetskaya Entsiklopediya,Moscow,1977. McLachlan,N.W.,Theory and Application of Mathieu Functions,Clarendon Press,Oxford,1947.Meixner,J.and Sch¨a fke,F.,Mathieusche Funktionen und Sph¨a roidfunktionnen,Springer-Verlag, Berlin,1965.Mikhlin,S.G.,Variational Methods in Mathematical Physics[in Russian],Nauka,Moscow,1970. Miles,J.W.,Integral Transforms in Applied Mathematics,Cambridge Univ.Press,Cambridge, 1971.Miller,W.(Jr.),Symmetry and Separation of Variables,Addison-Wesley,London,1977. Miller,J.(Jr.)and Rubel,L.A.,Functional separation of variables for Laplace equations in two dimensions,J.Phys.A,V ol.26,No.8,pp.1901–1913,1993.Moon,P.and Spencer,D.,Field Theory Handbook,Springer-Verlag,Berlin,1961.Morse,P.M.and Feshbach,H.,Methods of Theoretical Physics,Vols.1–2,McGraw-Hill,New York,1953.Murphy,G.M.,Ordinary Differential Equations and Their Solutions,D.Van Nostrand,New York, 1960.Myint-U,T.and Debnath,L.,Partial differential equations for scientists and engineers,North-Holland Publ.,New York,1987.Naimark,M.A.,Linear Differential Operators[in Russian],Nauka,Moscow,1969. Niederer,U.,The maximal kinematical invariance group of the harmonic oscillator,Helv.Phys.Acta,V ol.46,pp.191–200,1973.Nikiforov,A.F.and Uvarov,V.B.,Special Functions of Mathematical Physics.A Unified Introduction with Applications,Birkhauser Verlag,Basel-Boston,1988.Novikov,E.A.,Concerning turbulent diffusion in a stream with a transverse gradient of velosity, Appl.Math.Mech.(PMM),V ol.22,No.3,p.412–414,1958.Nusselt,W.,Abh¨a ngigkeit der W¨a rme¨u bergangzahl con der Rohr¨a nge,VDI Zeitschrift,Bd.54, No.28,S.1154–1158,1910.Olver,P.J.,Application of Lie Groups to Differential Equations,Springer-Verlag,New York,1986. Ovsiannikov,L.V.,Group Analysis of Differential Equations,Academic Press,New York,1982. Pavlovskii,Yu.N.,Analysis of some invariant solutions of boundary layer equations,Zh.Bych.Matem.i Mat.Fiziki,V ol.1,No.2,pp.280–294,1961.Petrovsky,I.G.,Lectures on Partial Differential Equations,Dover Publ.,New York,1991. Pinsky,M.A.,Introduction to Partial Differential Equations with Applications,McGraw-Hill,New York,1984.Polozhii,G.N.,Mathematical Physics Equations[in Russian],Vysshaya Shkola,Moscow,1964. Polyanin,A.D.,The structure of solutions of linear nonstationary boundary-value problems of mechanics and mathematical physics,Doklady Physics,V ol.45,No.8,pp.415–418,2000a. Polyanin,A.D.,Partial separation of variables in unsteady problems of mechanics and mathematical physics,Doklady Physics,V ol.45,No.12,pp.680–684,2000b.Polyanin,A.D.,Linear problems of heat and mass transfer:general relations and results,Theor.Found.Chem.Eng.,V ol.34,No.6,pp.509–520,2000c.Polyanin,A.D.,Handbook of Linear Mathematical Physics Equations[in Russian],Fizmatlit, Moscow,2001a.Polyanin,A.D.,Transformations and exact solutions of boundary layer equations with arbitrary functions,Doklady AN,V ol.379,No.3,2001b.Polyanin,A.D.,Exact solutions and transformations of the equations of a stationary laminar boundary layer,Theor.Found.Chem.Eng.,V ol.35,No.4,pp.319–328,2001c. Polyanin,A.D.,Generalized separable solutions of Navier–Stokes equations,Doklady AN,V ol.380, No.4,2001d.Polyanin,A.D.and Dilman,V.V.,Methods of Modeling Equations and Analogies in Chemical Engineering,CRC Press,Boca Raton,1994.Polyanin,A.D.,Kutepov,A.M.,Vyazmin,A.V.,and Kazenin,D.A.,Hydrodynamics,Mass and Heat Transfer in Chemical Engineering,Gordon&Breach Sci.Publ.,London,2001. Polyanin,A.D.and Manzhirov,A.V.,Handbook of Integral Equations,CRC Press,Boca Raton, 1998.Polyanin,A.D.,Vyazmin,A.V.,Zhurov,A.I.,and Kazenin,D.A.,Handbook of Exact Solutions of Heat and Mass Transfer Equations[in Russian],Faktorial,Moscow,1998.Polyanin,A.D.and Zaitsev,V.F.,Handbook of Exact Solutions for Ordinary Differential Equations, CRC Press,Boca Raton,1995.Polyanin,A.D.,Zaitsev,V.F.,and Moussiaux,A.,Handbook of First Order Partial Differential Equations,Gordon&Breach,London,2001.Polyanin,A.D.and Zhurov,A.I.,Exact solutions to nonlinear equations of mechanics and mathematical physics,Doklady Physics,V ol.43,No.6,pp.381–385,1998.Polyanin,A.D.,Zhurov,A.I.,and Vyazmin,A.V.,Generalized separation of variables in nonlinear heat and mass transfer equations,J.Non-Equilibrium Thermodynamics,V ol.25, No.3/4,pp.251–267,2000.Prudnikov,A.P.,Brychkov,Yu.A.,and Marichev,O.I.,Integrals and Series,Vol.1,Elementary Functions,Gordon&Breach Sci.Publ.,New York,1986.Prudnikov,A.P.,Brychkov,Yu.A.,and Marichev,O.I.,Integrals and Series,Vol.2,Special Functions,Gordon&Breach Sci.Publ.,New York,1986.Pukhnachev,V.V.,Group properties of the Navier–Stokes equations in the plane case,Zh.Prikl.Mekh.i Tekhn.Fiziki,No.1,pp.83–90,1960.Rimmer,P.L.,Heat transfer from a sphere in a stream of small Reynolds number,J.Fluid Mech., V ol.32,No.1,pp.1–7,1968.Rotem,Z.,and Neilson,J.E.,Exact solution for diffusion toflow down an incline,Can.J.Chem.Engng.,V ol.47,pp.341–346,1966.Schlichting,H.,Boundary Layer Theory,McGraw-Hill,New York,1981.Sedov,L.I.,Plane Problems of Hydrodynamics and Airdynamics[in Russian],Nauka,Moscow, 1980.Shilov,G.E.,Mathematical Analysis:A Second Special Course[in Russian],Nauka,Moscow,1965. Smirnov,V.I.,A Course of Higher Mathematics.Vols.2–3[in Russian],Nauka,Moscow,1974. Smirnov,M.M.,Second Order Partial Differential Equations[in Russian],Nauka,Moscow,1964. Smirnov,M.M.,Problems on Mathematical Physics Equations[in Russian],Nauka,Moscow, 1975.Sneddon,I.,Fourier Transformations,McGraw-Hill,New York,1951.Stakgold,I.,Boundary Value Problems of Mathematical Physics.Vols.I,II,SIAM,Philadelphia, 2000.Strauss,W.A.,Partial Differential Equations.An Introduction,John Wiley&Sons,New York, 1992.Sutton,W.G.L.,On the equation of diffusion in a turbulent medium,Proc.Poy.Soc.,Ser.A, V ol.138,No.988,pp.48–75,1943.Svirshchevskii,S.R.,Lie–B¨a cklund symmetries of linear ODEs and generalized separation of variables in nonlinear equations,Phys.Letters A,V ol.199,pp.344–348,1995.Taylor,M.,Partial Differential Equations,Vol.3,Springer-Verlag,New York,1996. Temme,N.M.,Special Functions.An Introduction to the Classical Functions of Mathematical Physics,Wiley-Interscience Publ.,New York,1996.Tikhonov,A.N.and Samarskii,A.A.,Equations of Mathematical Physics,Dover Publ.,New York,1990.Thomas,H.C.,Heterogeneous ion exchange in aflowing system,J.Amer.Chem.Soc.,V ol.66, pp.1664–1666,1944.Tomotika,S.and Tamada,K.,Studies on two-dimensional transonicflows of compressiblefluid, Part1,Quart.Appl.Math.,V ol.7,p.381,1950.Urvin,K.and Arscott,F.,Theory of the Whittaker–Hill equation,Proc.Roy.Soc.,V ol.A69, pp.28–44,1970.Vereshchagina,L.I.,Groupfibering of the spatial nonstationary boundary layer equations,Vestnik LGU,V ol.13,No.3,pp.82–86,1973.Vladimirov,V.S.,Mikhailov,V.P.,Vasharin A.A.,et al.,Collection of Problems on Mathematical Physics Equations[in Russian],Nauka,Moscow,1974.Vladimirov,V.S.,Mathematical Physics Equations[in Russian],Nauka,Moscow,1988. Vvedensky,D.,Partial Differential Equations,Addison-Wesley,Wakingham,1993. Whittaker,E.T.and Watson,G.N.,A Course of Modern Analysis,Vols.1–2,Cambridge Univ.Press,Cambridge,1952.Zachmanoglou,E.C.and Thoe,D.W.,Introduction to Partial Differential Equations with Applications,Dover Publ.,New York,1986.Zaitsev,V.F.and Polyanin,A.D.,Handbook of Partial Differential Equations:Exact Solutions[in Russian],MP Obrazovaniya,Moscow,1996.Zauderer,E.,Partial Differential Equations of Applied Mathematics,Wiley–Interscience Publ., New York,1989.Zhdanov,R.Z.,Separation of variables in the non-linear wave equation,J.Phys.A,V ol.27, pp.L291–L297,1994.Zwillinger,D.,Handbook of Differential Equations,Academic Press,San Diego,1998.。

morozov 原理
《Morozov原理》是一种关于互联网和科技影响社会的原则。

该原则由白俄罗斯作家埃万德罗·莫罗佐夫（EvgenyMorozov）提出，
旨在指出科技并不一定能够解决所有的社会问题，反而可能会加剧某些问题。

Morozov认为，科技的发展和应用需要对它们的社会影响加以思考和监管，而不是盲目地相信科技能够解决所有问题。

他警告人们不要过分乐观地认为科技会自动地消除社会不公或是提高民主，因为科技本身并不是价值中立的，它会受到各种利益和权力的影响。

Morozov的理论挑战了许多科技乐观主义者的观点，他认为智能手机、社交网络等科技的普及并没有使人们更加自由和平等，反而在某些方面对人们的自由和隐私造成了威胁。

他同时呼吁政府和公民社会应该对科技公司进行更加严格的监管，以保护公众利益和民主价值。

总的来说，《Morozov原理》提醒我们，在科技快速发展的时代，我们需要对科技的应用和影响进行深入思考，以便更好地应对科技带来的挑战和机遇。

- 1 -。