SuperGLUE

["SuperGLUE\ub294 2019\ub144 Google, DeepMind, \ub274\uc695\ub300\ud559\uad50 \uc5f0\uad6c\uc9c4\uc774 \uc790\uc5f0\uc5b4 \uc774\ud574(NLU) \uc2dc\uc2a4\ud15c\uc758 \ub2a5\ub825\uc744 \ud3c9\uac00\ud558\uae30 \uc704\ud574 \ub3c4\uc785\ud55c \ubca4\uce58\ub9c8\ud06c \ubaa8\uc74c\uc774\ub2e4. \uc774\ub294 \uace0\uc131\ub2a5 \ubaa8\ub378\ub85c \ud3ec\ud654 \uc0c1\ud0dc\uc5d0 \uc774\ub978 \ucd08\uae30 GLUE(\uc77c\ubc18 \uc5b8\uc5b4 \uc774\ud574 \ud3c9\uac00) \ubca4\uce58\ub9c8\ud06c\ubcf4\ub2e4 \ub354 \ub3c4\uc804\uc801\uc778 \ud6c4\uc18d \ubc84\uc804\uc73c\ub85c \uc124\uacc4\ub418\uc5c8\ub2e4. SuperGLUE\ub294 \uc0c1\ud638\ucc38\uc870 \ud574\uacb0, \uc9c8\uc758\uc751\ub2f5, \ud14d\uc2a4\ud2b8 \ud568\uc758, \uc5ec\ub7ec \ubb38\uc7a5\uc5d0 \uac78\uce5c \ucd94\ub860 \ub2a5\ub825\uc744 \uc2dc\ud5d8\ud558\ub294 \uc5ec\ub35f \uac00\uc9c0 \ub2e4\uc591\ud55c \uacfc\uc81c\ub85c \uad6c\uc131\ub418\uba70, \ub354 \uae4a\uc740 \uc5b8\uc5b4\uc801 \uc774\ud574\uc640 \uc0c1\uc2dd \uc9c0\uc2dd\uc744 \uc694\uad6c\ud558\ub294 \uacfc\uc81c\uc5d0 \uc911\uc810\uc744 \ub454\ub2e4.\n\n\uc774 \ubca4\uce58\ub9c8\ud06c\ub294 \ud2b9\ud788 Transformer (architecture) \uc544\ud0a4\ud14d\ucc98\uc5d0 \uae30\ubc18\ud55c Machine learning \ubc0f Deep learning \ubaa8\ub378\uc758 \uae09\uc18d\ud55c \ubc1c\uc804\uc5d0 \ub300\uc751\ud558\uc5ec \ub9cc\ub4e4\uc5b4\uc84c\uc73c\uba70, \uc774\ub7ec\ud55c \ubaa8\ub378\ub4e4\uc740 GLUE\uc5d0\uc11c \uc778\uac04\uc5d0 \uac00\uae4c\uc6b4 \uc131\ub2a5\uc744 \ub2ec\uc131\ud588\ub2e4. SuperGLUE\ub294 \ub2e4\uc911 \ubb38\uc7a5 \uc804\uc81c\ub97c \ud3ec\ud568\ud55c \ud14d\uc2a4\ud2b8 \ud568\uc758 \uc778\uc2dd\uacfc \ub2e4\ub2e8\uacc4 \ucd94\ub860\uc774 \ud544\uc694\ud55c \uc9c8\ubb38\uc5d0 \ub2f5\ud558\ub294 \ub4f1 \uae30\uacc4\uc5d0\uac8c \ub354 \uc5b4\ub824\uc6b4 \uacfc\uc81c\ub97c \ud3ec\ud568\ud558\uc5ec \uae30\uc900\uc744 \ub192\uc778\ub2e4. \uc774 \ubaa8\uc74c\uc740 \ubaa8\ub4e0 \uacfc\uc81c\uc758 \uc131\ub2a5\uc744 \uc885\ud569\ud55c \ub2e8\uc77c \uc810\uc218\ub97c \uc81c\uacf5\ud558\uc5ec \uc11c\ub85c \ub2e4\ub978 \ubaa8\ub378\uacfc \uc811\uadfc\ubc95\uc758 \uc9c1\uc811 \ube44\uad50\ub97c \uac00\ub2a5\ud558\uac8c \ud55c\ub2e4.\n\n## \uacfc\uc81c \uad6c\uc131\n\nSuperGLUE\ub294 \uac01\uac01 \uc5b8\uc5b4 \uc774\ud574\uc758 \ub69c\ub837\ud55c \uce21\uba74\uc744 \uaca8\ub0e5\ud55c \uc5ec\ub35f \uac00\uc9c0 \uacfc\uc81c\ub97c \ud3ec\ud568\ud55c\ub2e4. \uc774\ub7ec\ud55c \uacfc\uc81c\ub294 BoolQ(\uc608/\uc544\ub2c8\uc624 \ub2f5\ubcc0\uc744 \uc694\uad6c\ud558\ub294 \ubd88\ub9ac\uc5b8 \uc9c8\ubb38), CB(\uc57d\uc18d \uc740\ud589, \ud14d\uc2a4\ud2b8 \ud568\uc758 \uacfc\uc81c), COPA(\uadf8\ub7f4\ub4ef\ud55c \ub300\uc548 \uc120\ud0dd, \uc778\uacfc \ucd94\ub860 \uacfc\uc81c), MultiRC(\ub2e4\uc911 \ubb38\uc7a5 \ub3c5\ud574), ReCoRD(\uc0c1\uc2dd \ucd94\ub860\uc744 \ud3ec\ud568\ud55c \ub3c5\ud574), RTE(\ud14d\uc2a4\ud2b8 \ud568\uc758 \uc778\uc2dd), WiC(\ubb38\ub9e5 \uc18d \ub2e8\uc5b4, \uc5b4\ud718 \uc911\uc758\uc131 \ud574\uc18c \uacfc\uc81c), WSC(\uc704\ub178\uadf8\ub77c\ub4dc \uc2a4\ud0a4\ub9c8 \ucc4c\ub9b0\uc9c0, \ub300\uba85\uc0ac \ud574\uacb0 \uacfc\uc81c)\uc774\ub2e4. \uac01 \uacfc\uc81c\ub294 \ud45c\uba74\uc801 \ud328\ud134\uc5d0 \uc758\uc874\ud558\ub294 \ubaa8\ub378\uc5d0\uac8c \ub3c4\uc804\uc801\uc774\ub3c4\ub85d \uc124\uacc4\ub418\uc5c8\uc73c\uba70, \ub300\uc2e0 \uad6c\ubb38, \uc758\ubbf8\ub860, \uc138\uacc4 \uc9c0\uc2dd\uc5d0 \ub300\ud55c \uc774\ud574\ub97c \uc694\uad6c\ud55c\ub2e4.\n\n\uac10\uc815 \ubd84\uc11d\uacfc \uac19\uc740 \ub2e8\uc77c \ubb38\uc7a5 \uacfc\uc81c\ub97c \ud3ec\ud568\ud588\ub358 GLUE\uc640 \ub2ec\ub9ac, SuperGLUE\ub294 \uc5ec\ub7ec \ubb38\uc7a5\uacfc \ubcf5\uc7a1\ud55c \ucd94\ub860\uc744 \ud3ec\ud568\ud558\ub294 \uacfc\uc81c\ub97c \uac15\uc870\ud55c\ub2e4. \uc608\ub97c \ub4e4\uc5b4, WSC \uacfc\uc81c\ub294 \ubbf8\ubb18\ud55c \ub9e5\ub77d \ub2e8\uc11c\uc5d0 \uae30\ubc18\ud55c \ub300\uba85\uc0ac \ud574\uacb0\uc744 \uc694\uad6c\ud558\uba70, COPA\ub294 \uc804\uc81c\uac00 \uc8fc\uc5b4\uc84c\uc744 \ub54c \ub450 \ub300\uc548 \uc911 \uc5b4\ub290 \uac83\uc774 \ub354 \uadf8\ub7f4\ub4ef\ud55c\uc9c0 \ubb3b\ub294 \uc778\uacfc \ucd94\ub860\uc744 \uc2dc\ud5d8\ud55c\ub2e4. \uc774\ub7ec\ud55c \uacfc\uc81c\ub294 \uae30\uc874 \ub370\uc774\ud130\uc14b\uc5d0\uc11c \uac00\uc838\uc654\uc9c0\ub9cc, SuperGLUE\ub294 \ud45c\uc900\ud654\ub41c \ud6c8\ub828, \uac80\uc99d, \ud14c\uc2a4\ud2b8 \ubd84\ud560\uc744 \uac16\ucd98 \ud1b5\ud569 \ud3c9\uac00 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \uc81c\uacf5\ud55c\ub2e4.\n\n## \uc810\uc218 \uc0b0\uc815 \ubc0f \ud3c9\uac00\n\nSuperGLUE\ub294 \uac01 \uacfc\uc81c\uc758 \uc9c0\ud45c \ud3c9\uade0\uc73c\ub85c \uacc4\uc0b0\ub418\ub294 \ub2e8\uc77c \uc885\ud569 \uc810\uc218\ub97c \uc0ac\uc6a9\ud55c\ub2e4. \ub300\ubd80\ubd84\uc758 \uacfc\uc81c\uc5d0\uc11c \uc9c0\ud45c\ub294 \uc815\ud655\ub3c4\uc774\uc9c0\ub9cc, MultiRC\uc640 ReCoRD\uc758 \uacbd\uc6b0 \ub2e4\uc911 \ub2f5\ubcc0 \uc0c1\ud669\uc5d0\uc11c \ubd80\ubd84 \uc810\uc218\ub97c \ubc18\uc601\ud558\uae30 \uc704\ud574 F1 \uc810\uc218\ub97c \uc0ac\uc6a9\ud55c\ub2e4. \ubca4\uce58\ub9c8\ud06c\ub294 \ub610\ud55c \uc778\uac04 \uc131\uacfc \uae30\uc900\uc120\uc744 \ud3ec\ud568\ud558\uba70, \uc774\ub294 \uc778\uac04 \uc8fc\uc11d\uc790\uac00 \ubaa8\ub378\uacfc \uc720\uc0ac\ud55c \uc870\uac74\uc5d0\uc11c \uacfc\uc81c\ub97c \uc644\ub8cc\ud558\uc5ec \uc124\uc815\ub418\uc5c8\ub2e4. \uc774 \uae30\uc900\uc120\uc740 \uc9c4\ud589 \uc0c1\ud669\uc744 \uce21\uc815\ud558\ub294 \ucc38\uc870\uc810 \uc5ed\ud560\uc744 \ud55c\ub2e4.\n\n\uacfc\uc801\ud569\uc744 \ubc29\uc9c0\ud558\uae30 \uc704\ud574 \ud14c\uc2a4\ud2b8 \uc138\ud2b8\ub294 \ube44\uacf5\uac1c\ub85c \uc720\uc9c0\ub418\uba70, \uc81c\ucd9c\ubb3c\uc740 SuperGLUE \uc6f9\uc0ac\uc774\ud2b8\uc5d0 \ud638\uc2a4\ud305\ub41c \ub9ac\ub354\ubcf4\ub4dc\ub97c \ud1b5\ud574 \ud3c9\uac00\ub41c\ub2e4. \ubaa8\ub378\uc740 \ucd94\uac00 \ud6c8\ub828 \ub370\uc774\ud130\ub97c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc9c0\ub9cc, \ubca4\uce58\ub9c8\ud06c\ub294 \uc678\ubd80 \ub370\uc774\ud130 \uc720\ubb34\uc5d0 \ub530\ub978 \uacb0\uacfc\ub97c \ubcf4\uace0\ud558\uc5ec \uacf5\uc815\ud55c \ube44\uad50\ub97c \uc7a5\ub824\ud55c\ub2e4. \ud3c9\uac00 \ud504\ub85c\ud1a0\ucf5c\uc5d0\ub294 \ubd80\uc815, \uc591\ud654, \uc0c1\ud638\ucc38\uc870\uc640 \uac19\uc740 \uc5b8\uc5b4\uc801 \ud604\uc0c1\uc5d0 \uac78\uce5c \ubaa8\ub378 \ub2a5\ub825\uc758 \uc138\ubd80 \ubd84\uc11d\uc744 \uc81c\uacf5\ud558\ub294 \uc9c4\ub2e8 \uc138\ud2b8\ub3c4 \ud3ec\ud568\ub41c\ub2e4.\n\n## AI \uc5f0\uad6c\uc5d0 \ubbf8\uce5c \uc601\ud5a5\n\nSuperGLUE\ub294 \ub354 \uacac\uace0\ud55c \uc5b8\uc5b4 \ubaa8\ub378 \uac1c\ubc1c\uc744 \ucd94\uc9c4\ud568\uc73c\ub85c\uc368 Artificial intelligence \ubd84\uc57c\uc5d0 \uc0c1\ub2f9\ud55c \uc601\ud5a5\uc744 \ubbf8\ucce4\ub2e4. \ucd9c\uc2dc \uc9c1\ud6c4 BERT\uc640 \uadf8 \ud6c4\uc18d \ubaa8\ub378\ub4e4\uc774 SuperGLUE\uc5d0 \ub300\ud574 \ubca4\uce58\ub9c8\ud0b9\ub418\uc5c8\uc73c\uba70, \uc774\ub294 \uc544\ud0a4\ud14d\ucc98 \uac1c\uc120\uc744 \ucd09\uc9c4\ud55c \uc131\ub2a5 \uaca9\ucc28\ub97c \ub4dc\ub7ec\ub0c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, Large language model\uc778 GPT-2\uc640 \uc774\ud6c4 GPT-3\uc758 \ub3c4\uc785\uc740 \ubaa8\ub378 \ud06c\uae30\uc640 \ud6c8\ub828 \ub370\uc774\ud130\uc758 \ud655\uc7a5\uc774 SuperGLUE \uacfc\uc81c\uc5d0\uc11c \uc0c1\ub2f9\ud55c \uc774\ub4dd\uc744 \uac00\uc838\uc62c \uc218 \uc788\uc74c\uc744 \ubcf4\uc5ec\uc8fc\uc5c8\uc9c0\ub9cc, \uac00\uc7a5 \ud070 \ubaa8\ub378\uc870\ucc28\ub3c4 \ucc98\uc74c\uc5d0\ub294 \uc5ec\ub7ec \uacfc\uc81c\uc5d0\uc11c \uc778\uac04 \uc131\ub2a5\uc5d0 \ubbf8\uce58\uc9c0 \ubabb\ud588\ub2e4.\n\n\uc774 \ubca4\uce58\ub9c8\ud06c\ub294 \ub610\ud55c \uc0c8\ub85c\uc6b4 \ud6c8\ub828 \ubaa9\ud45c\uc640 \ubaa8\ub378 \uc544\ud0a4\ud14d\ucc98 \uc124\uacc4\uc5d0 \uc601\ud5a5\uc744 \uc8fc\uc5c8\ub2e4. Google DeepMind \ubc0f OpenAI\uc640 \uac19\uc740 \uae30\uad00\uc758 \uc5f0\uad6c\uc790\ub4e4\uc740 \ub2e4\uc911 \uacfc\uc81c \ud559\uc2b5, \uc801\ub300\uc801 \ud6c8\ub828, \uc678\ubd80 \uc9c0\uc2dd \ud1b5\ud569\uacfc \uac19\uc740 \ud601\uc2e0\uc744 \ud3c9\uac00\ud558\uae30 \uc704\ud574 SuperGLUE\ub97c \uc0ac\uc6a9\ud588\ub2e4. 2021\ub144\uae4c\uc9c0 T5\uc640 DeBERTa\ub97c \ud3ec\ud568\ud55c \uc5ec\ub7ec \ubaa8\ub378\uc774 \uc778\uac04 \uae30\uc900\uc120\uc744 \ucd08\uacfc\ud558\ub294 \uc810\uc218\ub97c \ub2ec\uc131\ud558\uc5ec \ubca4\uce58\ub9c8\ud06c\uac00 \ud3ec\ud654 \uc0c1\ud0dc\uc5d0 \uc774\ub974\ub800\uc74c\uc744 \ub098\ud0c0\ub0c8\ub2e4. \uc774\ub294 \ub354 \ubcf5\uc7a1\ud55c \ucd94\ub860\uacfc \uc0dd\uc131 \uacfc\uc81c\uc5d0 \ucd08\uc810\uc744 \ub9de\ucd98 SuperGLUE\uc758 \ud6c4\uc18d \ubc84\uc804\uacfc \uac19\uc740 \ub354 \uc5b4\ub824\uc6b4 \ubca4\uce58\ub9c8\ud06c \uac1c\ubc1c\ub85c \uc774\uc5b4\uc84c\ub2e4.\n\n## \ud55c\uacc4 \ubc0f \ube44\ud310\n\n\uc131\uacf5\uc5d0\ub3c4 \ubd88\uad6c\ud558\uace0 SuperGLUE\ub294 \ube44\ud310\uc5d0 \uc9c1\uba74\ud588\ub2e4. \uc77c\ubd80 \uc5f0\uad6c\uc790\ub4e4\uc740 \ubca4\uce58\ub9c8\ud06c\uc758 \uacfc\uc81c\uac00 \ub3c4\uc804\uc801\uc774\uae30\ub294 \ud558\uc9c0\ub9cc \uc8fc\ub85c \uac1d\uad00\uc2dd \ub610\ub294 \ubd84\ub958 \ubb38\uc81c\uc774\uae30 \ub54c\ubb38\uc5d0 \uc2e4\uc81c \uc138\uacc4\uc758 \uc5b8\uc5b4 \uc774\ud574\ub97c \uc644\uc804\ud788 \ud3ec\ucc29\ud558\uc9c0 \ubabb\ud55c\ub2e4\uace0 \uc8fc\uc7a5\ud55c\ub2e4. \uc8fc\uc694 \uc9c0\ud45c\ub85c \uc815\ud655\ub3c4\uc5d0 \uc758\uc874\ud558\ub294 \uac83\uc740 \ubcf4\uc815\uc774\ub098 \uc801\ub300\uc801 \uc785\ub825\uc5d0 \ub300\ud55c \uacac\uace0\uc131\uacfc \uac19\uc740 \ubaa8\ub378 \ud589\ub3d9\uc758 \ucc28\uc774\ub97c \ubaa8\ud638\ud558\uac8c \ub9cc\ub4e4 \uc218 \uc788\ub2e4. \ub610\ud55c \ubca4\uce58\ub9c8\ud06c\uc758 \uc601\uc5b4 \uc804\uc6a9 \ucd08\uc810\uc740 \ub2e4\uad6d\uc5b4 \ud658\uacbd\uc5d0 \ub300\ud55c \uc801\uc6a9 \uac00\ub2a5\uc131\uc744 \uc81c\ud55c\ud558\uba70, \uc774\ub294 \uc774\ud6c4 XGLUE\uc640 \uac19\uc740 \ubca4\uce58\ub9c8\ud06c\uac00 \ud574\uacb0\ud558\ub824\uace0 \ud55c \uaca9\ucc28\uc774\ub2e4.\n\n\ub610 \ub2e4\ub978 \uc6b0\ub824\ub294 \ubaa8\ub378\uc774 \ub370\uc774\ud130\uc14b\uc758 \uc778\uacf5\ubb3c\uc774\ub098 \ud3b8\ud5a5\uc744 \uc545\uc6a9\ud558\uc5ec \uc758\ub3c4\ub41c \ucd94\ub860\uc744 \uc218\ud589\ud558\uc9c0 \uc54a\uace0\ub3c4 \ubd80\ud480\ub824\uc9c4 \uc810\uc218\ub97c \uc5bb\uc744 \uac00\ub2a5\uc131\uc774\ub2e4. \uc608\ub97c \ub4e4\uc5b4, SuperGLUE\uc758 \uc77c\ubd80 \uacfc\uc81c\ub294 \ubaa8\ub378\uc774 \uc758\ub3c4\ub41c \ucd94\ub860 \uc5c6\uc774 \ud65c\uc6a9\ud560 \uc218 \uc788\ub294 \ud5c8\uc704 \uc0c1\uad00\uad00\uacc4\ub97c \ud3ec\ud568\ud558\ub294 \uac83\uc73c\ub85c \ubc1d\ud600\uc84c\ub2e4. \uc774\ub294 \ub354 \uc5c4\uaca9\ud55c \ud3c9\uac00 \ud504\ub85c\ud1a0\ucf5c\uacfc \ud2b9\uc815 \ub2a5\ub825\uc744 \uc870\uc0ac\ud558\ub294 \uc9c4\ub2e8 \ub370\uc774\ud130\uc14b \uac1c\ubc1c\uc5d0 \ub300\ud55c \uc694\uad6c\ub97c \ucd09\ubc1c\ud588\ub2e4.\n\n## \uc720\uc0b0 \ubc0f \ud6c4\uc18d \ubc84\uc804\n\nSuperGLUE\ub294 natural language processing \ucee4\ubba4\ub2c8\ud2f0\uc5d0\uc11c \ub110\ub9ac \uc778\uc6a9\ub418\ub294 \ubca4\uce58\ub9c8\ud06c\ub85c \ub0a8\uc544 \uc788\uc73c\uba70, NLU \uc2dc\uc2a4\ud15c \ud3c9\uac00\uc758 \ud45c\uc900 \ucc38\uc870 \uc5ed\ud560\uc744 \ud55c\ub2e4. \ub2e4\uc591\ud55c \uacfc\uc81c \uc0ac\uc6a9\uacfc \ube44\uacf5\uac1c \ud14c\uc2a4\ud2b8 \uc138\ud2b8\uc640 \uac19\uc740 \uc124\uacc4 \uc6d0\uce59\uc740 GLUE\uc758 \ud6c4\uc18d \ubc84\uc804, SuperGLUE \uc790\uccb4\uc758 \ud6c4\uc18d \ubc84\uc804, \uadf8\ub9ac\uace0 \ub2e4\uc911 \uacfc\uc81c \ud3c9\uac00 \ubaa8\uc74c\uc758 \uad11\ubc94\uc704\ud55c \ucd94\uc138\uc5d0 \uc601\ud5a5\uc744 \uc8fc\uc5c8\ub2e4. \uc774 \ubca4\uce58\ub9c8\ud06c\ub294 \ub610\ud55c SuperGLUE \uacfc\uc81c \uc131\ub2a5\uacfc \ubaa8\ub378 \ud06c\uae30 \uac04\uc758 \uc0c1\uad00\uad00\uacc4\uc5d0\uc11c \uc785\uc99d\ub41c \ubc14\uc640 \uac19\uc774 \ubaa8\ub378 \ud655\uc7a5\uc5d0 \ub300\ud55c \uad11\ubc94\uc704\ud55c \uc774\ud574\uc5d0 \uae30\uc5ec\ud588\ub2e4.\n\n2023\ub144 \ud604\uc7ac SuperGLUE\ub294 \uc131\uc219\ud55c \ubca4\uce58\ub9c8\ud06c\ub85c \uac04\uc8fc\ub418\uba70, \ub9ce\uc740 \ucd5c\ucca8\ub2e8 \ubaa8\ub378\uc774 \uc778\uac04 \uc131\ub2a5\uc744 \ucd08\uacfc\ud55c\ub2e4. \uadf8 \uc720\uc0b0\uc740 Neural network \uc5f0\uad6c\ub97c \ubc1c\uc804\uc2dc\ud0a4\ub294 \ub370 \uc788\uc5b4 \ub3c4\uc804\uc801\uc778 \ud3c9\uac00 \ubaa8\uc74c\uc758 \uc911\uc694\uc131\uc744 \uc785\uc99d\ud558\uace0, \ubaa8\ub378 \ub2a5\ub825\uacfc \ud568\uaed8 \uc9c4\ud654\ud558\ub294 \ubca4\uce58\ub9c8\ud06c\uc758 \ud544\uc694\uc131\uc744 \uac15\uc870\ud55c \ub370 \uc788\ub2e4. \ud5a5\ud6c4 \ubca4\uce58\ub9c8\ud06c\ub294 SuperGLUE\uac00 \ub9c8\ub828\ud55c \uae30\ubc18 \uc704\uc5d0 \ub354 \ub9ce\uc740 \uc0dd\uc131 \uacfc\uc81c, \ub2e4\uad6d\uc5b4 \ub370\uc774\ud130, \uc0c1\ud638\uc791\uc6a9 \ud658\uacbd\uc744 \ud1b5\ud569\ud560 \uac00\ub2a5\uc131\uc774 \ub192\ub2e4.", true]